2023-07-12 19:10:15,847 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-07-12 19:10:16,046 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3 2023-07-12 19:10:16,060 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-12 19:10:16,061 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518, deleteOnExit=true 2023-07-12 19:10:16,061 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-12 19:10:16,062 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/test.cache.data in system properties and HBase conf 2023-07-12 19:10:16,062 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/hadoop.tmp.dir in system properties and HBase conf 2023-07-12 19:10:16,062 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/hadoop.log.dir in system properties and HBase conf 2023-07-12 19:10:16,063 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-12 19:10:16,063 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-12 19:10:16,064 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-12 19:10:16,162 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-12 19:10:16,474 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-12 19:10:16,478 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-12 19:10:16,478 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-12 19:10:16,479 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-12 19:10:16,479 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-12 19:10:16,479 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-12 19:10:16,480 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-12 19:10:16,480 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-12 19:10:16,480 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-12 19:10:16,481 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-12 19:10:16,481 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/nfs.dump.dir in system properties and HBase conf 2023-07-12 19:10:16,481 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/java.io.tmpdir in system properties and HBase conf 2023-07-12 19:10:16,482 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-12 19:10:16,482 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-12 19:10:16,482 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-12 19:10:16,901 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-12 19:10:16,905 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-12 19:10:17,133 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-12 19:10:17,265 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-12 19:10:17,277 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 19:10:17,309 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 19:10:17,339 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/java.io.tmpdir/Jetty_localhost_localdomain_39349_hdfs____.gu5duc/webapp 2023-07-12 19:10:17,470 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:39349 2023-07-12 19:10:17,479 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-12 19:10:17,480 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-12 19:10:17,859 WARN [Listener at localhost.localdomain/43855] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 19:10:17,916 WARN [Listener at localhost.localdomain/43855] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-12 19:10:17,931 WARN [Listener at localhost.localdomain/43855] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 19:10:17,936 INFO [Listener at localhost.localdomain/43855] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 19:10:17,940 INFO [Listener at localhost.localdomain/43855] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/java.io.tmpdir/Jetty_localhost_39495_datanode____.dzdmgf/webapp 2023-07-12 19:10:18,015 INFO [Listener at localhost.localdomain/43855] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39495 2023-07-12 19:10:18,269 WARN [Listener at localhost.localdomain/44267] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 19:10:18,278 WARN [Listener at localhost.localdomain/44267] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-12 19:10:18,281 WARN [Listener at localhost.localdomain/44267] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 19:10:18,283 INFO [Listener at localhost.localdomain/44267] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 19:10:18,290 INFO [Listener at localhost.localdomain/44267] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/java.io.tmpdir/Jetty_localhost_43341_datanode____ob5g8o/webapp 2023-07-12 19:10:18,376 INFO [Listener at localhost.localdomain/44267] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43341 2023-07-12 19:10:18,385 WARN [Listener at localhost.localdomain/44537] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 19:10:18,403 WARN [Listener at localhost.localdomain/44537] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-12 19:10:18,406 WARN [Listener at localhost.localdomain/44537] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 19:10:18,407 INFO [Listener at localhost.localdomain/44537] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 19:10:18,412 INFO [Listener at localhost.localdomain/44537] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/java.io.tmpdir/Jetty_localhost_34945_datanode____.ie5h5s/webapp 2023-07-12 19:10:18,501 INFO [Listener at localhost.localdomain/44537] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34945 2023-07-12 19:10:18,514 WARN [Listener at localhost.localdomain/40405] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 19:10:18,670 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe227df911fd5566b: Processing first storage report for DS-76956236-ecfa-464a-ac60-47dd38a190c0 from datanode 20bbb2fd-7c0b-4edb-836c-47234fa474c0 2023-07-12 19:10:18,672 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe227df911fd5566b: from storage DS-76956236-ecfa-464a-ac60-47dd38a190c0 node DatanodeRegistration(127.0.0.1:43193, datanodeUuid=20bbb2fd-7c0b-4edb-836c-47234fa474c0, infoPort=35743, infoSecurePort=0, ipcPort=44537, storageInfo=lv=-57;cid=testClusterID;nsid=1185365847;c=1689189016963), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-07-12 19:10:18,672 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x56fd8eed9263808: Processing first storage report for DS-9cdc4cf3-6458-4aee-af86-5d18d78e0480 from datanode 1de994d2-d43c-471e-8302-bc00c9b9674d 2023-07-12 19:10:18,672 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x56fd8eed9263808: from storage DS-9cdc4cf3-6458-4aee-af86-5d18d78e0480 node DatanodeRegistration(127.0.0.1:41715, datanodeUuid=1de994d2-d43c-471e-8302-bc00c9b9674d, infoPort=44751, infoSecurePort=0, ipcPort=44267, storageInfo=lv=-57;cid=testClusterID;nsid=1185365847;c=1689189016963), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 19:10:18,672 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x53d414472e531ac6: Processing first storage report for DS-1f9fc0f1-1938-482c-8d6a-21117d1e4d68 from datanode 9225ce29-b06f-40b1-aaea-24784a80c6f4 2023-07-12 19:10:18,672 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x53d414472e531ac6: from storage DS-1f9fc0f1-1938-482c-8d6a-21117d1e4d68 node DatanodeRegistration(127.0.0.1:35483, datanodeUuid=9225ce29-b06f-40b1-aaea-24784a80c6f4, infoPort=41025, infoSecurePort=0, ipcPort=40405, storageInfo=lv=-57;cid=testClusterID;nsid=1185365847;c=1689189016963), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 19:10:18,673 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe227df911fd5566b: Processing first storage report for DS-ff40bff5-4153-49d2-8923-4ef33c259906 from datanode 20bbb2fd-7c0b-4edb-836c-47234fa474c0 2023-07-12 19:10:18,673 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe227df911fd5566b: from storage DS-ff40bff5-4153-49d2-8923-4ef33c259906 node DatanodeRegistration(127.0.0.1:43193, datanodeUuid=20bbb2fd-7c0b-4edb-836c-47234fa474c0, infoPort=35743, infoSecurePort=0, ipcPort=44537, storageInfo=lv=-57;cid=testClusterID;nsid=1185365847;c=1689189016963), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 19:10:18,673 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x56fd8eed9263808: Processing first storage report for DS-d4140245-ea8d-4850-b89b-859e9408d2f8 from datanode 1de994d2-d43c-471e-8302-bc00c9b9674d 2023-07-12 19:10:18,673 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x56fd8eed9263808: from storage DS-d4140245-ea8d-4850-b89b-859e9408d2f8 node DatanodeRegistration(127.0.0.1:41715, datanodeUuid=1de994d2-d43c-471e-8302-bc00c9b9674d, infoPort=44751, infoSecurePort=0, ipcPort=44267, storageInfo=lv=-57;cid=testClusterID;nsid=1185365847;c=1689189016963), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 19:10:18,673 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x53d414472e531ac6: Processing first storage report for DS-d77b7850-20e8-4402-98f6-eb06de5f541b from datanode 9225ce29-b06f-40b1-aaea-24784a80c6f4 2023-07-12 19:10:18,673 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x53d414472e531ac6: from storage DS-d77b7850-20e8-4402-98f6-eb06de5f541b node DatanodeRegistration(127.0.0.1:35483, datanodeUuid=9225ce29-b06f-40b1-aaea-24784a80c6f4, infoPort=41025, infoSecurePort=0, ipcPort=40405, storageInfo=lv=-57;cid=testClusterID;nsid=1185365847;c=1689189016963), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 19:10:18,849 DEBUG [Listener at localhost.localdomain/40405] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3 2023-07-12 19:10:18,903 INFO [Listener at localhost.localdomain/40405] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518/zookeeper_0, clientPort=63235, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-12 19:10:18,914 INFO [Listener at localhost.localdomain/40405] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=63235 2023-07-12 19:10:18,923 INFO [Listener at localhost.localdomain/40405] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 19:10:18,926 INFO [Listener at localhost.localdomain/40405] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 19:10:19,544 INFO [Listener at localhost.localdomain/40405] util.FSUtils(471): Created version file at hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99 with version=8 2023-07-12 19:10:19,545 INFO [Listener at localhost.localdomain/40405] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/hbase-staging 2023-07-12 19:10:19,778 INFO [Listener at localhost.localdomain/40405] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-12 19:10:20,142 INFO [Listener at localhost.localdomain/40405] client.ConnectionUtils(127): master/jenkins-hbase20:0 server-side Connection retries=45 2023-07-12 19:10:20,167 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 19:10:20,168 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 19:10:20,168 INFO [Listener at localhost.localdomain/40405] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 19:10:20,168 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 19:10:20,168 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 19:10:20,293 INFO [Listener at localhost.localdomain/40405] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 19:10:20,354 DEBUG [Listener at localhost.localdomain/40405] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-12 19:10:20,432 INFO [Listener at localhost.localdomain/40405] ipc.NettyRpcServer(120): Bind to /148.251.75.209:43565 2023-07-12 19:10:20,441 INFO [Listener at localhost.localdomain/40405] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 19:10:20,443 INFO [Listener at localhost.localdomain/40405] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 19:10:20,462 INFO [Listener at localhost.localdomain/40405] zookeeper.RecoverableZooKeeper(93): Process identifier=master:43565 connecting to ZooKeeper ensemble=127.0.0.1:63235 2023-07-12 19:10:20,494 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:435650x0, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 19:10:20,496 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:43565-0x100829769890000 connected 2023-07-12 19:10:20,516 DEBUG [Listener at localhost.localdomain/40405] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-12 19:10:20,517 DEBUG [Listener at localhost.localdomain/40405] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 19:10:20,522 DEBUG [Listener at localhost.localdomain/40405] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-12 19:10:20,531 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43565 2023-07-12 19:10:20,531 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43565 2023-07-12 19:10:20,531 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43565 2023-07-12 19:10:20,532 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43565 2023-07-12 19:10:20,532 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43565 2023-07-12 19:10:20,537 INFO [Listener at localhost.localdomain/40405] master.HMaster(444): hbase.rootdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99, hbase.cluster.distributed=false 2023-07-12 19:10:20,597 INFO [Listener at localhost.localdomain/40405] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-07-12 19:10:20,597 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 19:10:20,597 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 19:10:20,597 INFO [Listener at localhost.localdomain/40405] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 19:10:20,597 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 19:10:20,597 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 19:10:20,601 INFO [Listener at localhost.localdomain/40405] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 19:10:20,604 INFO [Listener at localhost.localdomain/40405] ipc.NettyRpcServer(120): Bind to /148.251.75.209:45655 2023-07-12 19:10:20,606 INFO [Listener at localhost.localdomain/40405] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-12 19:10:20,611 DEBUG [Listener at localhost.localdomain/40405] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-12 19:10:20,612 INFO [Listener at localhost.localdomain/40405] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 19:10:20,613 INFO [Listener at localhost.localdomain/40405] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 19:10:20,615 INFO [Listener at localhost.localdomain/40405] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45655 connecting to ZooKeeper ensemble=127.0.0.1:63235 2023-07-12 19:10:20,620 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:456550x0, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 19:10:20,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:45655-0x100829769890001 connected 2023-07-12 19:10:20,621 DEBUG [Listener at localhost.localdomain/40405] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-12 19:10:20,622 DEBUG [Listener at localhost.localdomain/40405] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 19:10:20,623 DEBUG [Listener at localhost.localdomain/40405] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-12 19:10:20,623 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45655 2023-07-12 19:10:20,624 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45655 2023-07-12 19:10:20,624 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45655 2023-07-12 19:10:20,624 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45655 2023-07-12 19:10:20,624 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45655 2023-07-12 19:10:20,635 INFO [Listener at localhost.localdomain/40405] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-07-12 19:10:20,636 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 19:10:20,636 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 19:10:20,636 INFO [Listener at localhost.localdomain/40405] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 19:10:20,636 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 19:10:20,636 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 19:10:20,637 INFO [Listener at localhost.localdomain/40405] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 19:10:20,638 INFO [Listener at localhost.localdomain/40405] ipc.NettyRpcServer(120): Bind to /148.251.75.209:33255 2023-07-12 19:10:20,639 INFO [Listener at localhost.localdomain/40405] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-12 19:10:20,641 DEBUG [Listener at localhost.localdomain/40405] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-12 19:10:20,642 INFO [Listener at localhost.localdomain/40405] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 19:10:20,644 INFO [Listener at localhost.localdomain/40405] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 19:10:20,647 INFO [Listener at localhost.localdomain/40405] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33255 connecting to ZooKeeper ensemble=127.0.0.1:63235 2023-07-12 19:10:20,651 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:332550x0, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 19:10:20,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:33255-0x100829769890002 connected 2023-07-12 19:10:20,653 DEBUG [Listener at localhost.localdomain/40405] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-12 19:10:20,654 DEBUG [Listener at localhost.localdomain/40405] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 19:10:20,655 DEBUG [Listener at localhost.localdomain/40405] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-12 19:10:20,656 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33255 2023-07-12 19:10:20,656 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33255 2023-07-12 19:10:20,657 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33255 2023-07-12 19:10:20,657 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33255 2023-07-12 19:10:20,657 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33255 2023-07-12 19:10:20,668 INFO [Listener at localhost.localdomain/40405] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-07-12 19:10:20,668 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 19:10:20,668 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 19:10:20,668 INFO [Listener at localhost.localdomain/40405] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 19:10:20,668 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 19:10:20,668 INFO [Listener at localhost.localdomain/40405] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 19:10:20,668 INFO [Listener at localhost.localdomain/40405] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 19:10:20,670 INFO [Listener at localhost.localdomain/40405] ipc.NettyRpcServer(120): Bind to /148.251.75.209:38353 2023-07-12 19:10:20,670 INFO [Listener at localhost.localdomain/40405] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-12 19:10:20,671 DEBUG [Listener at localhost.localdomain/40405] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-12 19:10:20,672 INFO [Listener at localhost.localdomain/40405] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 19:10:20,673 INFO [Listener at localhost.localdomain/40405] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 19:10:20,674 INFO [Listener at localhost.localdomain/40405] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38353 connecting to ZooKeeper ensemble=127.0.0.1:63235 2023-07-12 19:10:20,679 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:383530x0, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 19:10:20,680 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:38353-0x100829769890003 connected 2023-07-12 19:10:20,680 DEBUG [Listener at localhost.localdomain/40405] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-12 19:10:20,681 DEBUG [Listener at localhost.localdomain/40405] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 19:10:20,681 DEBUG [Listener at localhost.localdomain/40405] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-12 19:10:20,682 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38353 2023-07-12 19:10:20,682 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38353 2023-07-12 19:10:20,683 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38353 2023-07-12 19:10:20,683 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38353 2023-07-12 19:10:20,683 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38353 2023-07-12 19:10:20,685 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase20.apache.org,43565,1689189019651 2023-07-12 19:10:20,695 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-12 19:10:20,697 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase20.apache.org,43565,1689189019651 2023-07-12 19:10:20,713 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-12 19:10:20,713 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-12 19:10:20,713 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-12 19:10:20,713 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-12 19:10:20,714 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 19:10:20,715 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-12 19:10:20,717 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase20.apache.org,43565,1689189019651 from backup master directory 2023-07-12 19:10:20,717 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-12 19:10:20,720 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase20.apache.org,43565,1689189019651 2023-07-12 19:10:20,720 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-12 19:10:20,721 WARN [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 19:10:20,721 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase20.apache.org,43565,1689189019651 2023-07-12 19:10:20,723 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-12 19:10:20,725 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-12 19:10:20,813 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/hbase.id with ID: ed6779b5-6897-4a8d-a8c7-a39874f26e3e 2023-07-12 19:10:20,871 INFO [master/jenkins-hbase20:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 19:10:20,885 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 19:10:20,934 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x46e118df to 127.0.0.1:63235 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-12 19:10:20,965 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@13e0b13d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 19:10:20,983 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-12 19:10:20,985 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-12 19:10:20,998 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-07-12 19:10:20,998 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-07-12 19:10:21,000 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-12 19:10:21,004 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-12 19:10:21,004 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 19:10:21,032 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/MasterData/data/master/store-tmp 2023-07-12 19:10:21,117 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:21,117 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-12 19:10:21,117 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 19:10:21,117 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 19:10:21,118 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-12 19:10:21,118 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 19:10:21,118 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 19:10:21,118 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 19:10:21,120 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/MasterData/WALs/jenkins-hbase20.apache.org,43565,1689189019651 2023-07-12 19:10:21,139 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C43565%2C1689189019651, suffix=, logDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/MasterData/WALs/jenkins-hbase20.apache.org,43565,1689189019651, archiveDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/MasterData/oldWALs, maxLogs=10 2023-07-12 19:10:21,190 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41715,DS-9cdc4cf3-6458-4aee-af86-5d18d78e0480,DISK] 2023-07-12 19:10:21,190 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35483,DS-1f9fc0f1-1938-482c-8d6a-21117d1e4d68,DISK] 2023-07-12 19:10:21,190 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43193,DS-76956236-ecfa-464a-ac60-47dd38a190c0,DISK] 2023-07-12 19:10:21,198 DEBUG [RS-EventLoopGroup-5-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-12 19:10:21,257 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/MasterData/WALs/jenkins-hbase20.apache.org,43565,1689189019651/jenkins-hbase20.apache.org%2C43565%2C1689189019651.1689189021148 2023-07-12 19:10:21,257 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43193,DS-76956236-ecfa-464a-ac60-47dd38a190c0,DISK], DatanodeInfoWithStorage[127.0.0.1:35483,DS-1f9fc0f1-1938-482c-8d6a-21117d1e4d68,DISK], DatanodeInfoWithStorage[127.0.0.1:41715,DS-9cdc4cf3-6458-4aee-af86-5d18d78e0480,DISK]] 2023-07-12 19:10:21,258 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-12 19:10:21,258 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:21,261 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 19:10:21,262 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 19:10:21,313 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-12 19:10:21,320 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-12 19:10:21,341 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-12 19:10:21,351 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:21,357 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-12 19:10:21,359 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-12 19:10:21,373 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 19:10:21,378 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 19:10:21,379 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60252026, jitterRate=-0.10217484831809998}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 19:10:21,379 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 19:10:21,380 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-12 19:10:21,397 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:21,397 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:21,399 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:21,401 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:21,426 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 25 msec 2023-07-12 19:10:21,426 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-12 19:10:21,447 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-12 19:10:21,452 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-12 19:10:21,472 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-12 19:10:21,475 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-12 19:10:21,476 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-07-12 19:10:21,481 INFO [master/jenkins-hbase20:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-12 19:10:21,484 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-07-12 19:10:21,486 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 19:10:21,488 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-07-12 19:10:21,489 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-07-12 19:10:21,500 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-07-12 19:10:21,503 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-12 19:10:21,503 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-12 19:10:21,503 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-12 19:10:21,503 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-12 19:10:21,504 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 19:10:21,504 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase20.apache.org,43565,1689189019651, sessionid=0x100829769890000, setting cluster-up flag (Was=false) 2023-07-12 19:10:21,517 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 19:10:21,521 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-07-12 19:10:21,522 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,43565,1689189019651 2023-07-12 19:10:21,526 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 19:10:21,530 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-07-12 19:10:21,532 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,43565,1689189019651 2023-07-12 19:10:21,534 WARN [master/jenkins-hbase20:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.hbase-snapshot/.tmp 2023-07-12 19:10:21,587 INFO [RS:1;jenkins-hbase20:33255] regionserver.HRegionServer(951): ClusterId : ed6779b5-6897-4a8d-a8c7-a39874f26e3e 2023-07-12 19:10:21,587 INFO [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer(951): ClusterId : ed6779b5-6897-4a8d-a8c7-a39874f26e3e 2023-07-12 19:10:21,587 INFO [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer(951): ClusterId : ed6779b5-6897-4a8d-a8c7-a39874f26e3e 2023-07-12 19:10:21,593 DEBUG [RS:0;jenkins-hbase20:45655] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-12 19:10:21,593 DEBUG [RS:1;jenkins-hbase20:33255] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-12 19:10:21,593 DEBUG [RS:2;jenkins-hbase20:38353] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-12 19:10:21,599 DEBUG [RS:1;jenkins-hbase20:33255] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-12 19:10:21,599 DEBUG [RS:0;jenkins-hbase20:45655] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-12 19:10:21,599 DEBUG [RS:2;jenkins-hbase20:38353] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-12 19:10:21,599 DEBUG [RS:0;jenkins-hbase20:45655] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-12 19:10:21,599 DEBUG [RS:1;jenkins-hbase20:33255] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-12 19:10:21,599 DEBUG [RS:2;jenkins-hbase20:38353] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-12 19:10:21,602 DEBUG [RS:1;jenkins-hbase20:33255] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-12 19:10:21,602 DEBUG [RS:0;jenkins-hbase20:45655] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-12 19:10:21,602 DEBUG [RS:2;jenkins-hbase20:38353] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-12 19:10:21,604 DEBUG [RS:1;jenkins-hbase20:33255] zookeeper.ReadOnlyZKClient(139): Connect 0x5b5b3756 to 127.0.0.1:63235 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-12 19:10:21,604 DEBUG [RS:0;jenkins-hbase20:45655] zookeeper.ReadOnlyZKClient(139): Connect 0x5a50683e to 127.0.0.1:63235 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-12 19:10:21,604 DEBUG [RS:2;jenkins-hbase20:38353] zookeeper.ReadOnlyZKClient(139): Connect 0x03c2fe17 to 127.0.0.1:63235 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-12 19:10:21,613 DEBUG [RS:1;jenkins-hbase20:33255] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@559cc24d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 19:10:21,614 DEBUG [RS:2;jenkins-hbase20:38353] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@d05001f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 19:10:21,614 DEBUG [RS:1;jenkins-hbase20:33255] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@51a35c26, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-07-12 19:10:21,613 DEBUG [RS:0;jenkins-hbase20:45655] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@54979d04, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 19:10:21,614 DEBUG [RS:2;jenkins-hbase20:38353] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@93a9567, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-07-12 19:10:21,614 DEBUG [RS:0;jenkins-hbase20:45655] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@50a8034e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-07-12 19:10:21,635 DEBUG [RS:2;jenkins-hbase20:38353] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase20:38353 2023-07-12 19:10:21,637 DEBUG [RS:1;jenkins-hbase20:33255] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase20:33255 2023-07-12 19:10:21,639 DEBUG [RS:0;jenkins-hbase20:45655] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase20:45655 2023-07-12 19:10:21,640 INFO [RS:1;jenkins-hbase20:33255] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-12 19:10:21,640 INFO [RS:0;jenkins-hbase20:45655] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-12 19:10:21,641 INFO [RS:0;jenkins-hbase20:45655] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-12 19:10:21,640 INFO [RS:2;jenkins-hbase20:38353] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-12 19:10:21,641 INFO [RS:2;jenkins-hbase20:38353] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-12 19:10:21,641 DEBUG [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer(1022): About to register with Master. 2023-07-12 19:10:21,641 INFO [RS:1;jenkins-hbase20:33255] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-12 19:10:21,641 DEBUG [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer(1022): About to register with Master. 2023-07-12 19:10:21,641 DEBUG [RS:1;jenkins-hbase20:33255] regionserver.HRegionServer(1022): About to register with Master. 2023-07-12 19:10:21,643 INFO [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase20.apache.org,43565,1689189019651 with isa=jenkins-hbase20.apache.org/148.251.75.209:45655, startcode=1689189020596 2023-07-12 19:10:21,643 INFO [RS:1;jenkins-hbase20:33255] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase20.apache.org,43565,1689189019651 with isa=jenkins-hbase20.apache.org/148.251.75.209:33255, startcode=1689189020635 2023-07-12 19:10:21,643 INFO [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase20.apache.org,43565,1689189019651 with isa=jenkins-hbase20.apache.org/148.251.75.209:38353, startcode=1689189020667 2023-07-12 19:10:21,647 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:21,657 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-07-12 19:10:21,658 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-07-12 19:10:21,658 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-07-12 19:10:21,658 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-07-12 19:10:21,658 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase20:0, corePoolSize=10, maxPoolSize=10 2023-07-12 19:10:21,658 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,658 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-07-12 19:10:21,658 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,659 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189051659 2023-07-12 19:10:21,660 DEBUG [RS:0;jenkins-hbase20:45655] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-12 19:10:21,660 DEBUG [RS:1;jenkins-hbase20:33255] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-12 19:10:21,660 DEBUG [RS:2;jenkins-hbase20:38353] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-12 19:10:21,662 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-12 19:10:21,667 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:21,668 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-12 19:10:21,674 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-12 19:10:21,675 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-12 19:10:21,686 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-12 19:10:21,687 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-12 19:10:21,688 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-12 19:10:21,688 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-12 19:10:21,689 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,691 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-12 19:10:21,692 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-12 19:10:21,692 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-12 19:10:21,696 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-12 19:10:21,696 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-12 19:10:21,703 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1689189021697,5,FailOnTimeoutGroup] 2023-07-12 19:10:21,704 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1689189021704,5,FailOnTimeoutGroup] 2023-07-12 19:10:21,704 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,705 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-12 19:10:21,706 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,707 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,735 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-12 19:10:21,737 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-12 19:10:21,737 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99 2023-07-12 19:10:21,738 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:49131, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-12 19:10:21,738 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:57169, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-12 19:10:21,738 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:50363, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-12 19:10:21,755 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=43565] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:21,769 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:21,772 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-12 19:10:21,775 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/info 2023-07-12 19:10:21,775 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=43565] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:21,777 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=43565] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:21,777 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-12 19:10:21,779 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:21,779 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-12 19:10:21,782 DEBUG [RS:1;jenkins-hbase20:33255] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99 2023-07-12 19:10:21,782 DEBUG [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99 2023-07-12 19:10:21,782 DEBUG [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99 2023-07-12 19:10:21,783 DEBUG [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost.localdomain:43855 2023-07-12 19:10:21,783 DEBUG [RS:1;jenkins-hbase20:33255] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost.localdomain:43855 2023-07-12 19:10:21,783 DEBUG [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-12 19:10:21,783 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/rep_barrier 2023-07-12 19:10:21,783 DEBUG [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost.localdomain:43855 2023-07-12 19:10:21,783 DEBUG [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-12 19:10:21,783 DEBUG [RS:1;jenkins-hbase20:33255] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-12 19:10:21,784 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-12 19:10:21,785 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:21,785 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-12 19:10:21,788 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/table 2023-07-12 19:10:21,789 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-12 19:10:21,789 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-12 19:10:21,790 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:21,791 DEBUG [RS:1;jenkins-hbase20:33255] zookeeper.ZKUtil(162): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:21,791 DEBUG [RS:2;jenkins-hbase20:38353] zookeeper.ZKUtil(162): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:21,791 WARN [RS:1;jenkins-hbase20:33255] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 19:10:21,792 WARN [RS:2;jenkins-hbase20:38353] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 19:10:21,792 DEBUG [RS:0;jenkins-hbase20:45655] zookeeper.ZKUtil(162): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:21,792 INFO [RS:2;jenkins-hbase20:38353] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 19:10:21,792 INFO [RS:1;jenkins-hbase20:33255] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 19:10:21,792 WARN [RS:0;jenkins-hbase20:45655] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 19:10:21,793 DEBUG [RS:1;jenkins-hbase20:33255] regionserver.HRegionServer(1948): logDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/WALs/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:21,793 INFO [RS:0;jenkins-hbase20:45655] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 19:10:21,793 DEBUG [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer(1948): logDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/WALs/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:21,793 DEBUG [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer(1948): logDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/WALs/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:21,793 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,38353,1689189020667] 2023-07-12 19:10:21,793 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,45655,1689189020596] 2023-07-12 19:10:21,793 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,33255,1689189020635] 2023-07-12 19:10:21,794 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740 2023-07-12 19:10:21,797 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740 2023-07-12 19:10:21,807 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-12 19:10:21,810 DEBUG [RS:2;jenkins-hbase20:38353] zookeeper.ZKUtil(162): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:21,810 DEBUG [RS:0;jenkins-hbase20:45655] zookeeper.ZKUtil(162): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:21,810 DEBUG [RS:1;jenkins-hbase20:33255] zookeeper.ZKUtil(162): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:21,810 DEBUG [RS:2;jenkins-hbase20:38353] zookeeper.ZKUtil(162): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:21,810 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-12 19:10:21,811 DEBUG [RS:1;jenkins-hbase20:33255] zookeeper.ZKUtil(162): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:21,811 DEBUG [RS:0;jenkins-hbase20:45655] zookeeper.ZKUtil(162): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:21,811 DEBUG [RS:1;jenkins-hbase20:33255] zookeeper.ZKUtil(162): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:21,811 DEBUG [RS:2;jenkins-hbase20:38353] zookeeper.ZKUtil(162): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:21,812 DEBUG [RS:0;jenkins-hbase20:45655] zookeeper.ZKUtil(162): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:21,815 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 19:10:21,816 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69836439, jitterRate=0.04064403474330902}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-12 19:10:21,816 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-12 19:10:21,816 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-12 19:10:21,816 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-12 19:10:21,816 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-12 19:10:21,817 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-12 19:10:21,817 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-12 19:10:21,818 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-12 19:10:21,818 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-12 19:10:21,823 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:21,823 DEBUG [RS:0;jenkins-hbase20:45655] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-12 19:10:21,824 DEBUG [RS:1;jenkins-hbase20:33255] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-12 19:10:21,823 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-12 19:10:21,823 DEBUG [RS:2;jenkins-hbase20:38353] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-12 19:10:21,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-12 19:10:21,835 INFO [RS:0;jenkins-hbase20:45655] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-12 19:10:21,835 INFO [RS:1;jenkins-hbase20:33255] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-12 19:10:21,835 INFO [RS:2;jenkins-hbase20:38353] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-12 19:10:21,850 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-12 19:10:21,853 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-12 19:10:21,858 INFO [RS:1;jenkins-hbase20:33255] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-12 19:10:21,860 INFO [RS:2;jenkins-hbase20:38353] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-12 19:10:21,861 INFO [RS:0;jenkins-hbase20:45655] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-12 19:10:21,864 INFO [RS:1;jenkins-hbase20:33255] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-12 19:10:21,864 INFO [RS:0;jenkins-hbase20:45655] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-12 19:10:21,864 INFO [RS:2;jenkins-hbase20:38353] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-12 19:10:21,864 INFO [RS:0;jenkins-hbase20:45655] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,864 INFO [RS:1;jenkins-hbase20:33255] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,865 INFO [RS:2;jenkins-hbase20:38353] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,866 INFO [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-12 19:10:21,866 INFO [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-12 19:10:21,866 INFO [RS:1;jenkins-hbase20:33255] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-12 19:10:21,874 INFO [RS:1;jenkins-hbase20:33255] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,874 INFO [RS:0;jenkins-hbase20:45655] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,874 INFO [RS:2;jenkins-hbase20:38353] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,874 DEBUG [RS:1;jenkins-hbase20:33255] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,875 DEBUG [RS:2;jenkins-hbase20:38353] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,874 DEBUG [RS:0;jenkins-hbase20:45655] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,875 DEBUG [RS:2;jenkins-hbase20:38353] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,875 DEBUG [RS:1;jenkins-hbase20:33255] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,875 DEBUG [RS:2;jenkins-hbase20:38353] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,875 DEBUG [RS:1;jenkins-hbase20:33255] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,875 DEBUG [RS:2;jenkins-hbase20:38353] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,875 DEBUG [RS:1;jenkins-hbase20:33255] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,875 DEBUG [RS:2;jenkins-hbase20:38353] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,875 DEBUG [RS:1;jenkins-hbase20:33255] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,876 DEBUG [RS:2;jenkins-hbase20:38353] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-07-12 19:10:21,876 DEBUG [RS:1;jenkins-hbase20:33255] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-07-12 19:10:21,876 DEBUG [RS:2;jenkins-hbase20:38353] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,876 DEBUG [RS:1;jenkins-hbase20:33255] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,875 DEBUG [RS:0;jenkins-hbase20:45655] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,876 DEBUG [RS:1;jenkins-hbase20:33255] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,876 DEBUG [RS:2;jenkins-hbase20:38353] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,876 DEBUG [RS:1;jenkins-hbase20:33255] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,876 DEBUG [RS:2;jenkins-hbase20:38353] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,876 DEBUG [RS:1;jenkins-hbase20:33255] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,876 DEBUG [RS:2;jenkins-hbase20:38353] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,876 DEBUG [RS:0;jenkins-hbase20:45655] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,877 DEBUG [RS:0;jenkins-hbase20:45655] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,878 DEBUG [RS:0;jenkins-hbase20:45655] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,878 DEBUG [RS:0;jenkins-hbase20:45655] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-07-12 19:10:21,878 DEBUG [RS:0;jenkins-hbase20:45655] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,878 DEBUG [RS:0;jenkins-hbase20:45655] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,878 DEBUG [RS:0;jenkins-hbase20:45655] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,878 INFO [RS:1;jenkins-hbase20:33255] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,879 DEBUG [RS:0;jenkins-hbase20:45655] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 19:10:21,879 INFO [RS:1;jenkins-hbase20:33255] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,879 INFO [RS:1;jenkins-hbase20:33255] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,879 INFO [RS:2;jenkins-hbase20:38353] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,879 INFO [RS:2;jenkins-hbase20:38353] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,880 INFO [RS:0;jenkins-hbase20:45655] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,880 INFO [RS:2;jenkins-hbase20:38353] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,880 INFO [RS:0;jenkins-hbase20:45655] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,880 INFO [RS:0;jenkins-hbase20:45655] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,895 INFO [RS:1;jenkins-hbase20:33255] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-12 19:10:21,895 INFO [RS:2;jenkins-hbase20:38353] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-12 19:10:21,896 INFO [RS:0;jenkins-hbase20:45655] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-12 19:10:21,898 INFO [RS:2;jenkins-hbase20:38353] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,38353,1689189020667-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,898 INFO [RS:1;jenkins-hbase20:33255] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,33255,1689189020635-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,898 INFO [RS:0;jenkins-hbase20:45655] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,45655,1689189020596-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:21,919 INFO [RS:0;jenkins-hbase20:45655] regionserver.Replication(203): jenkins-hbase20.apache.org,45655,1689189020596 started 2023-07-12 19:10:21,919 INFO [RS:1;jenkins-hbase20:33255] regionserver.Replication(203): jenkins-hbase20.apache.org,33255,1689189020635 started 2023-07-12 19:10:21,919 INFO [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer(1637): Serving as jenkins-hbase20.apache.org,45655,1689189020596, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:45655, sessionid=0x100829769890001 2023-07-12 19:10:21,919 INFO [RS:2;jenkins-hbase20:38353] regionserver.Replication(203): jenkins-hbase20.apache.org,38353,1689189020667 started 2023-07-12 19:10:21,919 INFO [RS:1;jenkins-hbase20:33255] regionserver.HRegionServer(1637): Serving as jenkins-hbase20.apache.org,33255,1689189020635, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:33255, sessionid=0x100829769890002 2023-07-12 19:10:21,919 INFO [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer(1637): Serving as jenkins-hbase20.apache.org,38353,1689189020667, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:38353, sessionid=0x100829769890003 2023-07-12 19:10:21,920 DEBUG [RS:1;jenkins-hbase20:33255] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-12 19:10:21,920 DEBUG [RS:2;jenkins-hbase20:38353] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-12 19:10:21,920 DEBUG [RS:1;jenkins-hbase20:33255] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:21,920 DEBUG [RS:2;jenkins-hbase20:38353] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:21,920 DEBUG [RS:1;jenkins-hbase20:33255] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,33255,1689189020635' 2023-07-12 19:10:21,920 DEBUG [RS:2;jenkins-hbase20:38353] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,38353,1689189020667' 2023-07-12 19:10:21,920 DEBUG [RS:0;jenkins-hbase20:45655] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-12 19:10:21,921 DEBUG [RS:0;jenkins-hbase20:45655] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:21,921 DEBUG [RS:0;jenkins-hbase20:45655] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,45655,1689189020596' 2023-07-12 19:10:21,921 DEBUG [RS:0;jenkins-hbase20:45655] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:21,921 DEBUG [RS:2;jenkins-hbase20:38353] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:21,921 DEBUG [RS:1;jenkins-hbase20:33255] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:21,922 DEBUG [RS:2;jenkins-hbase20:38353] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:21,922 DEBUG [RS:0;jenkins-hbase20:45655] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:21,922 DEBUG [RS:1;jenkins-hbase20:33255] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:21,922 DEBUG [RS:2;jenkins-hbase20:38353] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-12 19:10:21,922 DEBUG [RS:0;jenkins-hbase20:45655] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-12 19:10:21,922 DEBUG [RS:2;jenkins-hbase20:38353] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-12 19:10:21,922 DEBUG [RS:1;jenkins-hbase20:33255] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-12 19:10:21,922 DEBUG [RS:2;jenkins-hbase20:38353] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:21,922 DEBUG [RS:0;jenkins-hbase20:45655] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-12 19:10:21,922 DEBUG [RS:2;jenkins-hbase20:38353] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,38353,1689189020667' 2023-07-12 19:10:21,923 DEBUG [RS:2;jenkins-hbase20:38353] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-12 19:10:21,922 DEBUG [RS:1;jenkins-hbase20:33255] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-12 19:10:21,922 DEBUG [RS:0;jenkins-hbase20:45655] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:21,923 DEBUG [RS:1;jenkins-hbase20:33255] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:21,923 DEBUG [RS:0;jenkins-hbase20:45655] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,45655,1689189020596' 2023-07-12 19:10:21,923 DEBUG [RS:0;jenkins-hbase20:45655] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-12 19:10:21,923 DEBUG [RS:1;jenkins-hbase20:33255] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,33255,1689189020635' 2023-07-12 19:10:21,923 DEBUG [RS:2;jenkins-hbase20:38353] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-12 19:10:21,923 DEBUG [RS:1;jenkins-hbase20:33255] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-12 19:10:21,924 DEBUG [RS:0;jenkins-hbase20:45655] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-12 19:10:21,924 DEBUG [RS:2;jenkins-hbase20:38353] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-12 19:10:21,924 DEBUG [RS:1;jenkins-hbase20:33255] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-12 19:10:21,924 INFO [RS:2;jenkins-hbase20:38353] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-12 19:10:21,924 INFO [RS:2;jenkins-hbase20:38353] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-12 19:10:21,924 DEBUG [RS:0;jenkins-hbase20:45655] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-12 19:10:21,924 INFO [RS:0;jenkins-hbase20:45655] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-12 19:10:21,925 DEBUG [RS:1;jenkins-hbase20:33255] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-12 19:10:21,925 INFO [RS:0;jenkins-hbase20:45655] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-12 19:10:21,925 INFO [RS:1;jenkins-hbase20:33255] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-12 19:10:21,925 INFO [RS:1;jenkins-hbase20:33255] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-12 19:10:22,006 DEBUG [jenkins-hbase20:43565] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-07-12 19:10:22,012 DEBUG [jenkins-hbase20:43565] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 19:10:22,018 DEBUG [jenkins-hbase20:43565] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 19:10:22,018 DEBUG [jenkins-hbase20:43565] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 19:10:22,018 DEBUG [jenkins-hbase20:43565] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 19:10:22,018 DEBUG [jenkins-hbase20:43565] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 19:10:22,020 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,38353,1689189020667, state=OPENING 2023-07-12 19:10:22,028 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-12 19:10:22,029 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 19:10:22,030 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-12 19:10:22,033 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:22,038 INFO [RS:2;jenkins-hbase20:38353] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C38353%2C1689189020667, suffix=, logDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/WALs/jenkins-hbase20.apache.org,38353,1689189020667, archiveDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/oldWALs, maxLogs=32 2023-07-12 19:10:22,038 INFO [RS:1;jenkins-hbase20:33255] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C33255%2C1689189020635, suffix=, logDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/WALs/jenkins-hbase20.apache.org,33255,1689189020635, archiveDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/oldWALs, maxLogs=32 2023-07-12 19:10:22,039 INFO [RS:0;jenkins-hbase20:45655] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C45655%2C1689189020596, suffix=, logDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/WALs/jenkins-hbase20.apache.org,45655,1689189020596, archiveDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/oldWALs, maxLogs=32 2023-07-12 19:10:22,061 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43193,DS-76956236-ecfa-464a-ac60-47dd38a190c0,DISK] 2023-07-12 19:10:22,064 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41715,DS-9cdc4cf3-6458-4aee-af86-5d18d78e0480,DISK] 2023-07-12 19:10:22,070 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35483,DS-1f9fc0f1-1938-482c-8d6a-21117d1e4d68,DISK] 2023-07-12 19:10:22,071 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35483,DS-1f9fc0f1-1938-482c-8d6a-21117d1e4d68,DISK] 2023-07-12 19:10:22,078 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43193,DS-76956236-ecfa-464a-ac60-47dd38a190c0,DISK] 2023-07-12 19:10:22,080 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41715,DS-9cdc4cf3-6458-4aee-af86-5d18d78e0480,DISK] 2023-07-12 19:10:22,081 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43193,DS-76956236-ecfa-464a-ac60-47dd38a190c0,DISK] 2023-07-12 19:10:22,081 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35483,DS-1f9fc0f1-1938-482c-8d6a-21117d1e4d68,DISK] 2023-07-12 19:10:22,081 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41715,DS-9cdc4cf3-6458-4aee-af86-5d18d78e0480,DISK] 2023-07-12 19:10:22,100 INFO [RS:1;jenkins-hbase20:33255] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/WALs/jenkins-hbase20.apache.org,33255,1689189020635/jenkins-hbase20.apache.org%2C33255%2C1689189020635.1689189022042 2023-07-12 19:10:22,100 INFO [RS:0;jenkins-hbase20:45655] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/WALs/jenkins-hbase20.apache.org,45655,1689189020596/jenkins-hbase20.apache.org%2C45655%2C1689189020596.1689189022042 2023-07-12 19:10:22,100 INFO [RS:2;jenkins-hbase20:38353] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/WALs/jenkins-hbase20.apache.org,38353,1689189020667/jenkins-hbase20.apache.org%2C38353%2C1689189020667.1689189022043 2023-07-12 19:10:22,100 DEBUG [RS:1;jenkins-hbase20:33255] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43193,DS-76956236-ecfa-464a-ac60-47dd38a190c0,DISK], DatanodeInfoWithStorage[127.0.0.1:35483,DS-1f9fc0f1-1938-482c-8d6a-21117d1e4d68,DISK], DatanodeInfoWithStorage[127.0.0.1:41715,DS-9cdc4cf3-6458-4aee-af86-5d18d78e0480,DISK]] 2023-07-12 19:10:22,100 DEBUG [RS:0;jenkins-hbase20:45655] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41715,DS-9cdc4cf3-6458-4aee-af86-5d18d78e0480,DISK], DatanodeInfoWithStorage[127.0.0.1:35483,DS-1f9fc0f1-1938-482c-8d6a-21117d1e4d68,DISK], DatanodeInfoWithStorage[127.0.0.1:43193,DS-76956236-ecfa-464a-ac60-47dd38a190c0,DISK]] 2023-07-12 19:10:22,102 DEBUG [RS:2;jenkins-hbase20:38353] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43193,DS-76956236-ecfa-464a-ac60-47dd38a190c0,DISK], DatanodeInfoWithStorage[127.0.0.1:41715,DS-9cdc4cf3-6458-4aee-af86-5d18d78e0480,DISK], DatanodeInfoWithStorage[127.0.0.1:35483,DS-1f9fc0f1-1938-482c-8d6a-21117d1e4d68,DISK]] 2023-07-12 19:10:22,222 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:22,224 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 19:10:22,230 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:59952, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 19:10:22,243 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-12 19:10:22,243 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 19:10:22,246 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C38353%2C1689189020667.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/WALs/jenkins-hbase20.apache.org,38353,1689189020667, archiveDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/oldWALs, maxLogs=32 2023-07-12 19:10:22,268 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43193,DS-76956236-ecfa-464a-ac60-47dd38a190c0,DISK] 2023-07-12 19:10:22,269 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41715,DS-9cdc4cf3-6458-4aee-af86-5d18d78e0480,DISK] 2023-07-12 19:10:22,271 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35483,DS-1f9fc0f1-1938-482c-8d6a-21117d1e4d68,DISK] 2023-07-12 19:10:22,278 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/WALs/jenkins-hbase20.apache.org,38353,1689189020667/jenkins-hbase20.apache.org%2C38353%2C1689189020667.meta.1689189022248.meta 2023-07-12 19:10:22,279 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43193,DS-76956236-ecfa-464a-ac60-47dd38a190c0,DISK], DatanodeInfoWithStorage[127.0.0.1:41715,DS-9cdc4cf3-6458-4aee-af86-5d18d78e0480,DISK], DatanodeInfoWithStorage[127.0.0.1:35483,DS-1f9fc0f1-1938-482c-8d6a-21117d1e4d68,DISK]] 2023-07-12 19:10:22,279 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-12 19:10:22,284 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:22,285 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-12 19:10:22,301 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-12 19:10:22,304 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-12 19:10:22,307 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-12 19:10:22,307 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:22,307 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-12 19:10:22,308 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-12 19:10:22,311 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-12 19:10:22,313 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/info 2023-07-12 19:10:22,313 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/info 2023-07-12 19:10:22,314 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-12 19:10:22,316 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:22,316 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-12 19:10:22,318 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/rep_barrier 2023-07-12 19:10:22,318 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/rep_barrier 2023-07-12 19:10:22,319 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-12 19:10:22,320 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:22,320 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-12 19:10:22,322 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/table 2023-07-12 19:10:22,322 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/table 2023-07-12 19:10:22,322 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-12 19:10:22,323 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:22,326 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740 2023-07-12 19:10:22,333 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740 2023-07-12 19:10:22,339 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-12 19:10:22,342 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-12 19:10:22,344 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61734338, jitterRate=-0.08008667826652527}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-12 19:10:22,344 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-12 19:10:22,357 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689189022216 2023-07-12 19:10:22,377 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-12 19:10:22,378 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-12 19:10:22,378 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,38353,1689189020667, state=OPEN 2023-07-12 19:10:22,381 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-12 19:10:22,381 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-12 19:10:22,386 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-12 19:10:22,386 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,38353,1689189020667 in 348 msec 2023-07-12 19:10:22,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-12 19:10:22,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 554 msec 2023-07-12 19:10:22,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 824 msec 2023-07-12 19:10:22,400 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689189022399, completionTime=-1 2023-07-12 19:10:22,400 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-07-12 19:10:22,400 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:22,462 DEBUG [hconnection-0x7acefbf8-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 19:10:22,465 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:59962, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 19:10:22,483 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:22,483 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189082483 2023-07-12 19:10:22,483 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189142483 2023-07-12 19:10:22,483 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 83 msec 2023-07-12 19:10:22,507 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,43565,1689189019651-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:22,507 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,43565,1689189019651-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:22,507 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,43565,1689189019651-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:22,508 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase20:43565, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:22,509 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-12 19:10:22,518 DEBUG [master/jenkins-hbase20:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-12 19:10:22,526 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-12 19:10:22,527 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-12 19:10:22,536 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:22,539 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 19:10:22,543 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 19:10:22,567 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/hbase/namespace/da4636671d8cf580eb751750528e08a9 2023-07-12 19:10:22,570 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/hbase/namespace/da4636671d8cf580eb751750528e08a9 empty. 2023-07-12 19:10:22,570 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/hbase/namespace/da4636671d8cf580eb751750528e08a9 2023-07-12 19:10:22,571 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-12 19:10:22,606 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-12 19:10:22,609 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => da4636671d8cf580eb751750528e08a9, NAME => 'hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp 2023-07-12 19:10:22,631 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:22,632 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing da4636671d8cf580eb751750528e08a9, disabling compactions & flushes 2023-07-12 19:10:22,632 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9. 2023-07-12 19:10:22,632 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9. 2023-07-12 19:10:22,632 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9. after waiting 0 ms 2023-07-12 19:10:22,632 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9. 2023-07-12 19:10:22,632 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9. 2023-07-12 19:10:22,632 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for da4636671d8cf580eb751750528e08a9: 2023-07-12 19:10:22,637 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 19:10:22,655 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689189022642"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189022642"}]},"ts":"1689189022642"} 2023-07-12 19:10:22,681 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 19:10:22,683 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 19:10:22,687 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189022683"}]},"ts":"1689189022683"} 2023-07-12 19:10:22,694 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-12 19:10:22,698 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 19:10:22,699 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 19:10:22,699 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 19:10:22,699 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 19:10:22,699 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 19:10:22,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=da4636671d8cf580eb751750528e08a9, ASSIGN}] 2023-07-12 19:10:22,705 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=da4636671d8cf580eb751750528e08a9, ASSIGN 2023-07-12 19:10:22,706 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=da4636671d8cf580eb751750528e08a9, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,45655,1689189020596; forceNewPlan=false, retain=false 2023-07-12 19:10:22,860 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 19:10:22,861 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=da4636671d8cf580eb751750528e08a9, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:22,861 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689189022861"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189022861"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189022861"}]},"ts":"1689189022861"} 2023-07-12 19:10:22,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure da4636671d8cf580eb751750528e08a9, server=jenkins-hbase20.apache.org,45655,1689189020596}] 2023-07-12 19:10:23,028 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:23,029 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 19:10:23,038 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52606, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 19:10:23,045 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9. 2023-07-12 19:10:23,047 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => da4636671d8cf580eb751750528e08a9, NAME => 'hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9.', STARTKEY => '', ENDKEY => ''} 2023-07-12 19:10:23,048 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:23,049 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace da4636671d8cf580eb751750528e08a9 2023-07-12 19:10:23,049 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:23,050 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for da4636671d8cf580eb751750528e08a9 2023-07-12 19:10:23,050 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for da4636671d8cf580eb751750528e08a9 2023-07-12 19:10:23,052 INFO [StoreOpener-da4636671d8cf580eb751750528e08a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region da4636671d8cf580eb751750528e08a9 2023-07-12 19:10:23,056 DEBUG [StoreOpener-da4636671d8cf580eb751750528e08a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/namespace/da4636671d8cf580eb751750528e08a9/info 2023-07-12 19:10:23,056 DEBUG [StoreOpener-da4636671d8cf580eb751750528e08a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/namespace/da4636671d8cf580eb751750528e08a9/info 2023-07-12 19:10:23,057 INFO [StoreOpener-da4636671d8cf580eb751750528e08a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region da4636671d8cf580eb751750528e08a9 columnFamilyName info 2023-07-12 19:10:23,058 INFO [StoreOpener-da4636671d8cf580eb751750528e08a9-1] regionserver.HStore(310): Store=da4636671d8cf580eb751750528e08a9/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:23,061 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/namespace/da4636671d8cf580eb751750528e08a9 2023-07-12 19:10:23,062 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/namespace/da4636671d8cf580eb751750528e08a9 2023-07-12 19:10:23,067 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for da4636671d8cf580eb751750528e08a9 2023-07-12 19:10:23,071 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/namespace/da4636671d8cf580eb751750528e08a9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 19:10:23,072 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened da4636671d8cf580eb751750528e08a9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75273017, jitterRate=0.12165535986423492}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 19:10:23,072 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for da4636671d8cf580eb751750528e08a9: 2023-07-12 19:10:23,074 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9., pid=6, masterSystemTime=1689189023028 2023-07-12 19:10:23,079 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9. 2023-07-12 19:10:23,080 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9. 2023-07-12 19:10:23,082 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=da4636671d8cf580eb751750528e08a9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:23,082 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689189023081"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189023081"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189023081"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189023081"}]},"ts":"1689189023081"} 2023-07-12 19:10:23,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-12 19:10:23,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure da4636671d8cf580eb751750528e08a9, server=jenkins-hbase20.apache.org,45655,1689189020596 in 214 msec 2023-07-12 19:10:23,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-12 19:10:23,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=da4636671d8cf580eb751750528e08a9, ASSIGN in 389 msec 2023-07-12 19:10:23,095 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 19:10:23,095 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189023095"}]},"ts":"1689189023095"} 2023-07-12 19:10:23,097 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-12 19:10:23,100 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 19:10:23,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 572 msec 2023-07-12 19:10:23,139 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-07-12 19:10:23,141 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-07-12 19:10:23,142 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 19:10:23,162 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 19:10:23,166 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52620, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 19:10:23,180 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:23,197 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-12 19:10:23,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 31 msec 2023-07-12 19:10:23,214 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:23,225 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-12 19:10:23,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-07-12 19:10:23,245 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-12 19:10:23,246 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-07-12 19:10:23,247 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.526sec 2023-07-12 19:10:23,250 INFO [master/jenkins-hbase20:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-12 19:10:23,252 INFO [master/jenkins-hbase20:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-12 19:10:23,252 INFO [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-12 19:10:23,254 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,43565,1689189019651-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-12 19:10:23,255 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,43565,1689189019651-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-12 19:10:23,266 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-12 19:10:23,294 DEBUG [Listener at localhost.localdomain/40405] zookeeper.ReadOnlyZKClient(139): Connect 0x33996176 to 127.0.0.1:63235 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-12 19:10:23,299 DEBUG [Listener at localhost.localdomain/40405] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6600db08, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 19:10:23,315 DEBUG [hconnection-0x49452fc5-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 19:10:23,329 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:59964, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 19:10:23,337 INFO [Listener at localhost.localdomain/40405] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase20.apache.org,43565,1689189019651 2023-07-12 19:10:23,367 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=361, OpenFileDescriptor=698, MaxFileDescriptor=60000, SystemLoadAverage=242, ProcessCount=181, AvailableMemoryMB=6087 2023-07-12 19:10:23,372 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:10:23,381 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-12 19:10:23,385 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:41996, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-12 19:10:23,398 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-07-12 19:10:23,398 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 19:10:23,399 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 19:10:23,405 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,408 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 19:10:23,410 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 19:10:23,413 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:23,414 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069 empty. 2023-07-12 19:10:23,416 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:23,416 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-07-12 19:10:23,440 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-07-12 19:10:23,442 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => da5689e107dc59e64b9df0b3d4d1e069, NAME => 'testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp 2023-07-12 19:10:23,461 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:23,461 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1604): Closing da5689e107dc59e64b9df0b3d4d1e069, disabling compactions & flushes 2023-07-12 19:10:23,461 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:23,461 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:23,462 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. after waiting 1 ms 2023-07-12 19:10:23,462 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:23,462 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:23,462 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1558): Region close journal for da5689e107dc59e64b9df0b3d4d1e069: 2023-07-12 19:10:23,466 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 19:10:23,468 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1689189023467"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189023467"}]},"ts":"1689189023467"} 2023-07-12 19:10:23,470 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 19:10:23,471 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 19:10:23,472 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189023471"}]},"ts":"1689189023471"} 2023-07-12 19:10:23,474 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-07-12 19:10:23,477 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 19:10:23,479 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 19:10:23,479 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 19:10:23,479 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 19:10:23,479 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 19:10:23,479 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=da5689e107dc59e64b9df0b3d4d1e069, ASSIGN}] 2023-07-12 19:10:23,482 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=da5689e107dc59e64b9df0b3d4d1e069, ASSIGN 2023-07-12 19:10:23,483 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=da5689e107dc59e64b9df0b3d4d1e069, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,38353,1689189020667; forceNewPlan=false, retain=false 2023-07-12 19:10:23,634 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 19:10:23,636 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=da5689e107dc59e64b9df0b3d4d1e069, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:23,637 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1689189023636"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189023636"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189023636"}]},"ts":"1689189023636"} 2023-07-12 19:10:23,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure da5689e107dc59e64b9df0b3d4d1e069, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:23,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:23,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => da5689e107dc59e64b9df0b3d4d1e069, NAME => 'testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069.', STARTKEY => '', ENDKEY => ''} 2023-07-12 19:10:23,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:23,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:23,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:23,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:23,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:23,810 INFO [StoreOpener-da5689e107dc59e64b9df0b3d4d1e069-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:23,813 DEBUG [StoreOpener-da5689e107dc59e64b9df0b3d4d1e069-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1 2023-07-12 19:10:23,813 DEBUG [StoreOpener-da5689e107dc59e64b9df0b3d4d1e069-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1 2023-07-12 19:10:23,813 INFO [StoreOpener-da5689e107dc59e64b9df0b3d4d1e069-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region da5689e107dc59e64b9df0b3d4d1e069 columnFamilyName cf1 2023-07-12 19:10:23,814 INFO [StoreOpener-da5689e107dc59e64b9df0b3d4d1e069-1] regionserver.HStore(310): Store=da5689e107dc59e64b9df0b3d4d1e069/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:23,814 INFO [StoreOpener-da5689e107dc59e64b9df0b3d4d1e069-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:23,816 DEBUG [StoreOpener-da5689e107dc59e64b9df0b3d4d1e069-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2 2023-07-12 19:10:23,816 DEBUG [StoreOpener-da5689e107dc59e64b9df0b3d4d1e069-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2 2023-07-12 19:10:23,817 INFO [StoreOpener-da5689e107dc59e64b9df0b3d4d1e069-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region da5689e107dc59e64b9df0b3d4d1e069 columnFamilyName cf2 2023-07-12 19:10:23,817 INFO [StoreOpener-da5689e107dc59e64b9df0b3d4d1e069-1] regionserver.HStore(310): Store=da5689e107dc59e64b9df0b3d4d1e069/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:23,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:23,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:23,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:10:23,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:23,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 19:10:23,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened da5689e107dc59e64b9df0b3d4d1e069; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62907637, jitterRate=-0.06260316073894501}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:10:23,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for da5689e107dc59e64b9df0b3d4d1e069: 2023-07-12 19:10:23,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069., pid=11, masterSystemTime=1689189023797 2023-07-12 19:10:23,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:23,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:23,861 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=da5689e107dc59e64b9df0b3d4d1e069, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:23,861 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1689189023861"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189023861"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189023861"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189023861"}]},"ts":"1689189023861"} 2023-07-12 19:10:23,867 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-12 19:10:23,867 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure da5689e107dc59e64b9df0b3d4d1e069, server=jenkins-hbase20.apache.org,38353,1689189020667 in 221 msec 2023-07-12 19:10:23,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-12 19:10:23,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=da5689e107dc59e64b9df0b3d4d1e069, ASSIGN in 389 msec 2023-07-12 19:10:23,873 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 19:10:23,874 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189023873"}]},"ts":"1689189023873"} 2023-07-12 19:10:23,876 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-07-12 19:10:23,879 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 19:10:23,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 476 msec 2023-07-12 19:10:23,930 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:23,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,938 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-12 19:10:23,938 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:23,939 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-12 19:10:23,939 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-12 19:10:23,940 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,940 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,941 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:23,941 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:23,941 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:23,941 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:23,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:23,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:23,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:23,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:23,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:23,942 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:23,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:23,942 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:23,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,942 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,942 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,942 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:23,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:23,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:23,943 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,943 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,943 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,945 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,945 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,945 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,945 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:23,945 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:23,945 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:23,945 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:23,945 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:23,945 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:23,947 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:23,946 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:23,947 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:23,947 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:23,947 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:23,947 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:23,947 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:23,948 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:23,948 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:23,948 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:23,948 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. started... 2023-07-12 19:10:23,950 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing da5689e107dc59e64b9df0b3d4d1e069 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 19:10:23,952 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,952 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,952 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,953 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:23,953 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:23,953 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:24,016 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf1/b69e5cab366442498cae15b556b89a62 2023-07-12 19:10:24,075 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf2/77afc58b907740b695d7177f54ef9aa6 2023-07-12 19:10:24,086 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf1/b69e5cab366442498cae15b556b89a62 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/b69e5cab366442498cae15b556b89a62 2023-07-12 19:10:24,095 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/b69e5cab366442498cae15b556b89a62, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:10:24,098 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf2/77afc58b907740b695d7177f54ef9aa6 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/77afc58b907740b695d7177f54ef9aa6 2023-07-12 19:10:24,109 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/77afc58b907740b695d7177f54ef9aa6, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:10:24,111 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for da5689e107dc59e64b9df0b3d4d1e069 in 161ms, sequenceid=6, compaction requested=false 2023-07-12 19:10:24,113 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-07-12 19:10:24,115 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for da5689e107dc59e64b9df0b3d4d1e069: 2023-07-12 19:10:24,115 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:24,115 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:24,116 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:24,116 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,116 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:24,116 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:24,118 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,118 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:24,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:24,118 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,118 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:24,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:24,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:24,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:24,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-12 19:10:24,121 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-12 19:10:24,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@574269da[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:24,121 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,123 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,123 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,123 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,124 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:24,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,124 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:24,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,124 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:24,124 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:24,124 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:24,124 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:24,124 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,124 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:24,124 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:24,124 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:24,124 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:24,126 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:24,126 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,126 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:24,127 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,127 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:24,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:24,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:24,127 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,127 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:24,127 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:24,127 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,128 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:24,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:24,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:24,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:24,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:24,132 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-12 19:10:24,132 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:24,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:10:24,133 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:24,133 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,133 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:24,137 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,137 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:24,137 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,137 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,137 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,138 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:24,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,138 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,138 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:24,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,139 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,139 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:24,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,141 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:24,141 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,142 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,161 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,161 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,161 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,161 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,161 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,161 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,161 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-12 19:10:24,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:24,161 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,161 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,162 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,162 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,162 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,163 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,163 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,163 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,163 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,163 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,163 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,164 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:24,164 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:24,264 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:24,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:24,303 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:24,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:24,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,307 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-12 19:10:24,307 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:24,308 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-12 19:10:24,308 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-12 19:10:24,308 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,308 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,309 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,309 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,309 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,309 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,310 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,310 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:24,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,310 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,310 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:24,311 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,311 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:24,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,311 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:24,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,311 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,313 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 19:10:24,313 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 19:10:24,313 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 19:10:24,313 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,313 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,313 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:24,313 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,313 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:24,313 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:24,314 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:24,313 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:24,314 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:24,314 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:24,314 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:24,314 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:24,314 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:24,315 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:24,315 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:24,315 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:24,315 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:24,315 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:24,315 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:24,316 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. started... 2023-07-12 19:10:24,316 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing da5689e107dc59e64b9df0b3d4d1e069 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:24,320 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,320 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,320 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,320 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:24,320 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,320 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:24,340 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf1/1827d2d3fc89457da9bf965aef32a83d 2023-07-12 19:10:24,366 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf2/18507120a48446ce918f7936d5c13a7a 2023-07-12 19:10:24,376 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf1/1827d2d3fc89457da9bf965aef32a83d as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/1827d2d3fc89457da9bf965aef32a83d 2023-07-12 19:10:24,384 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/1827d2d3fc89457da9bf965aef32a83d, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:10:24,386 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf2/18507120a48446ce918f7936d5c13a7a as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/18507120a48446ce918f7936d5c13a7a 2023-07-12 19:10:24,395 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/18507120a48446ce918f7936d5c13a7a, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:10:24,396 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for da5689e107dc59e64b9df0b3d4d1e069 in 80ms, sequenceid=19, compaction requested=false 2023-07-12 19:10:24,397 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for da5689e107dc59e64b9df0b3d4d1e069: 2023-07-12 19:10:24,397 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:24,397 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:24,397 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:24,397 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,397 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:24,397 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:24,399 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,399 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:24,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:24,399 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,399 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:24,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:24,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:24,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:24,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-12 19:10:24,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@eef4f20[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:24,402 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-12 19:10:24,402 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,403 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,403 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,403 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,403 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:24,403 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,403 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:24,403 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,403 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:24,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,403 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:24,403 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,403 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:24,404 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:24,404 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:24,404 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:24,405 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:24,405 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,405 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:24,406 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:24,406 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,406 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:24,407 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:24,407 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,407 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,407 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:24,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:24,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:24,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:24,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:10:24,412 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-12 19:10:24,412 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:24,412 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:24,412 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,412 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:24,420 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,420 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,420 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,420 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,420 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,420 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:24,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:24,421 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,420 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,421 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:24,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,422 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:24,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,424 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:24,424 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,425 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,432 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,432 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,432 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,432 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-12 19:10:24,432 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:24,432 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,432 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,432 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,433 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,433 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,432 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,434 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:24,434 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,434 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,434 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:24,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,434 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,434 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,434 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,435 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,435 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,435 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,435 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,534 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:24,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:24,570 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:24,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:24,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,573 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-12 19:10:24,573 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:24,574 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-12 19:10:24,574 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-12 19:10:24,574 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,574 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,576 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,576 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,576 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,576 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,576 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,576 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:24,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:24,577 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,577 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,578 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,578 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 19:10:24,578 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,578 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 19:10:24,578 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:24,578 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,578 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 19:10:24,578 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:24,578 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:24,578 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:24,578 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,579 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:24,579 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:24,579 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:24,579 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:24,580 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:24,580 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:24,579 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:24,580 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:24,580 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:24,580 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:24,581 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:24,581 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:24,581 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. started... 2023-07-12 19:10:24,581 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing da5689e107dc59e64b9df0b3d4d1e069 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:24,597 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,597 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,598 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,598 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:24,599 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,599 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:24,605 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf1/acc7023d5f734655b815d4db916bc879 2023-07-12 19:10:24,638 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf2/db767ea0c7b04a1296ade791afaab7c7 2023-07-12 19:10:24,650 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf1/acc7023d5f734655b815d4db916bc879 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/acc7023d5f734655b815d4db916bc879 2023-07-12 19:10:24,658 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/acc7023d5f734655b815d4db916bc879, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:10:24,659 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf2/db767ea0c7b04a1296ade791afaab7c7 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/db767ea0c7b04a1296ade791afaab7c7 2023-07-12 19:10:24,667 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/db767ea0c7b04a1296ade791afaab7c7, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:10:24,668 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for da5689e107dc59e64b9df0b3d4d1e069 in 87ms, sequenceid=32, compaction requested=true 2023-07-12 19:10:24,668 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for da5689e107dc59e64b9df0b3d4d1e069: 2023-07-12 19:10:24,668 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:24,668 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:24,670 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:24,670 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,670 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:24,670 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:24,672 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,672 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:24,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:24,672 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,672 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:24,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:24,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:24,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:24,675 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-12 19:10:24,675 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@26e47225[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:24,675 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-12 19:10:24,675 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,676 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,676 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,676 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,676 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:24,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,676 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:24,676 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:24,676 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:24,676 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:24,676 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,676 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:24,676 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:24,676 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:24,676 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:24,676 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:24,678 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:24,678 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,678 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:24,678 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,679 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:24,679 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:24,679 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:24,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:24,679 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:24,679 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,679 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:24,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:24,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:24,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:24,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:24,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:10:24,684 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-12 19:10:24,684 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:24,684 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:24,684 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,684 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:24,694 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,694 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,694 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,694 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,694 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,694 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:24,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,694 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,695 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:24,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,696 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,696 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:24,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:24,700 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,701 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,701 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,707 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,707 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,707 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,708 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,707 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,707 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,707 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,708 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,708 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-12 19:10:24,709 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:24,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,708 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,709 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,709 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:24,709 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,709 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:24,709 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,710 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,710 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,710 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,710 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,710 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,710 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,810 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:24,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:24,841 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:24,844 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:24,844 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,844 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-12 19:10:24,844 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:24,845 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-12 19:10:24,845 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-12 19:10:24,845 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,845 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,846 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,846 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,846 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,846 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,847 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,847 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:24,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,847 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,847 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,847 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,848 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,848 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,848 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,848 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 19:10:24,848 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 19:10:24,848 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 19:10:24,848 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,848 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,848 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:24,848 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,848 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:24,848 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:24,849 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:24,848 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:24,849 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:24,849 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:24,849 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:24,849 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:24,850 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:24,850 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:24,850 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:24,850 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:24,851 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:24,851 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:24,851 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:24,851 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. started... 2023-07-12 19:10:24,851 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing da5689e107dc59e64b9df0b3d4d1e069 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:24,851 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,852 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,852 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,852 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:24,852 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,853 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:24,876 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf1/650629c852ad4efeaf97a2e3f9684be3 2023-07-12 19:10:24,899 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf2/056ee9812a9647dfb4097f307dc577ef 2023-07-12 19:10:24,907 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf1/650629c852ad4efeaf97a2e3f9684be3 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/650629c852ad4efeaf97a2e3f9684be3 2023-07-12 19:10:24,915 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/650629c852ad4efeaf97a2e3f9684be3, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:10:24,916 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf2/056ee9812a9647dfb4097f307dc577ef as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/056ee9812a9647dfb4097f307dc577ef 2023-07-12 19:10:24,924 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/056ee9812a9647dfb4097f307dc577ef, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:10:24,925 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for da5689e107dc59e64b9df0b3d4d1e069 in 74ms, sequenceid=45, compaction requested=true 2023-07-12 19:10:24,925 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for da5689e107dc59e64b9df0b3d4d1e069: 2023-07-12 19:10:24,925 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:24,925 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:24,926 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:24,926 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,926 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:24,926 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:24,928 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,928 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:24,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:24,929 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,929 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:24,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:24,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:24,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:24,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-12 19:10:24,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4721513[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:24,931 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-12 19:10:24,931 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,932 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,932 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,932 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,932 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:24,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,932 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,933 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:24,933 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:24,932 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:24,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,933 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:24,933 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:24,933 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:24,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,933 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:24,933 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:24,933 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:24,934 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,934 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:24,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:24,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:24,934 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,934 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:24,934 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:24,935 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:24,935 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,935 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:24,935 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:24,935 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:24,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:24,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:24,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:24,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:10:24,939 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-12 19:10:24,939 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:24,939 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:24,939 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,939 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:24,940 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,941 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,941 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:24,941 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:24,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,941 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,941 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,941 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,941 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:24,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,942 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,942 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:24,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,943 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:24,944 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,944 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,946 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,946 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,946 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,946 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,946 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,946 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,946 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:24,946 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-12 19:10:24,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:24,946 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,946 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:24,946 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:24,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,946 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:24,947 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:24,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,948 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:24,948 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:24,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,948 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:24,948 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:24,948 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:24,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:24,948 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:24,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:24,949 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,048 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:25,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:25,079 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:25,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:25,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,081 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-12 19:10:25,081 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:25,082 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-12 19:10:25,082 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-12 19:10:25,083 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,083 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,084 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:25,084 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:25,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:25,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:25,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:25,084 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,084 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:25,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:25,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:25,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:25,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,085 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,085 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:25,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:25,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:25,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:25,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 19:10:25,086 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,086 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,086 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:25,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 19:10:25,086 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:25,088 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 19:10:25,088 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,089 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,089 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:25,089 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:25,089 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:25,089 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:25,090 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:25,089 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:25,090 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:25,090 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:25,090 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:25,091 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:25,091 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:25,092 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:25,092 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:25,092 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:25,092 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. started... 2023-07-12 19:10:25,092 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing da5689e107dc59e64b9df0b3d4d1e069 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:25,095 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,095 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,096 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,096 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:25,096 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,096 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:25,118 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf1/13441224975444eb9fa642ab4cc24060 2023-07-12 19:10:25,144 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf2/0095784930694e8ea8d0f7fbfed3b824 2023-07-12 19:10:25,151 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf1/13441224975444eb9fa642ab4cc24060 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/13441224975444eb9fa642ab4cc24060 2023-07-12 19:10:25,157 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/13441224975444eb9fa642ab4cc24060, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:10:25,158 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf2/0095784930694e8ea8d0f7fbfed3b824 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/0095784930694e8ea8d0f7fbfed3b824 2023-07-12 19:10:25,166 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/0095784930694e8ea8d0f7fbfed3b824, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:10:25,168 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for da5689e107dc59e64b9df0b3d4d1e069 in 76ms, sequenceid=58, compaction requested=true 2023-07-12 19:10:25,168 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for da5689e107dc59e64b9df0b3d4d1e069: 2023-07-12 19:10:25,168 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:25,168 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:25,169 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:25,169 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,169 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:25,169 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:25,170 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,170 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:25,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:25,171 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,171 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:25,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:25,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:25,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:25,173 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-12 19:10:25,173 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40bab0fe[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:25,173 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-12 19:10:25,173 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,174 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,174 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,174 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,174 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:25,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,174 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:25,175 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:25,175 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:25,174 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,175 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:25,175 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:25,175 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:25,175 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:25,175 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:25,175 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:25,176 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,176 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:25,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:25,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:25,176 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:25,176 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:25,176 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:25,176 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:25,177 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:25,177 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:25,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:25,177 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:25,178 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:25,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:25,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:25,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:25,183 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:10:25,183 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-12 19:10:25,183 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:25,183 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:25,183 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,183 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:25,186 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,186 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,186 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,186 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,186 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:25,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,186 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:25,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:25,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:25,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,187 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:25,187 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:25,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:25,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:25,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:25,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:25,188 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,188 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:25,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:25,190 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,191 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,191 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,196 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,196 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,196 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:25,196 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:25,196 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,196 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-12 19:10:25,196 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:25,196 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:25,196 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:25,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:25,197 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:25,197 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:25,197 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,197 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,197 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,197 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,198 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,198 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,198 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,198 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:25,198 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:25,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:25,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:25,199 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:25,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:25,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:25,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:25,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:25,199 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:25,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:25,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:25,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:25,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:25,297 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:25,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:25,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:25,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:25,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,329 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-12 19:10:25,329 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:25,330 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-12 19:10:25,330 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-12 19:10:25,330 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,330 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,331 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:25,331 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:25,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:25,331 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:25,331 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:25,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:25,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:25,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:25,332 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:25,332 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:25,332 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:25,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,333 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,333 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:25,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,333 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 19:10:25,333 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,333 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,334 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:25,334 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:25,334 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 19:10:25,334 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:25,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,334 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,335 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:25,335 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:25,335 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:25,335 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:25,335 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:25,335 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. started... 2023-07-12 19:10:25,336 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:25,336 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:25,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:25,336 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:25,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,336 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing da5689e107dc59e64b9df0b3d4d1e069 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:25,336 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,336 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 19:10:25,336 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,337 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 19:10:25,337 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:25,338 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 19:10:25,338 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:25,338 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:25,354 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,359 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf1/5f51d0d28a1943e5941215a0f0378418 2023-07-12 19:10:25,361 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,361 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,361 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:25,362 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,362 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:25,384 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf2/1e1bf654215d40e89fdfdf5ad54ab6f0 2023-07-12 19:10:25,393 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf1/5f51d0d28a1943e5941215a0f0378418 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/5f51d0d28a1943e5941215a0f0378418 2023-07-12 19:10:25,403 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/5f51d0d28a1943e5941215a0f0378418, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:10:25,404 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf2/1e1bf654215d40e89fdfdf5ad54ab6f0 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/1e1bf654215d40e89fdfdf5ad54ab6f0 2023-07-12 19:10:25,412 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/1e1bf654215d40e89fdfdf5ad54ab6f0, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:10:25,413 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for da5689e107dc59e64b9df0b3d4d1e069 in 77ms, sequenceid=71, compaction requested=true 2023-07-12 19:10:25,413 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for da5689e107dc59e64b9df0b3d4d1e069: 2023-07-12 19:10:25,413 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:25,413 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:25,414 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:25,414 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,414 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 19:10:25,414 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 19:10:25,415 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,415 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:25,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:25,415 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,415 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:25,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:25,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:25,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:25,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-12 19:10:25,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34586186[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:25,418 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-12 19:10:25,418 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,418 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,418 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,418 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,419 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:25,419 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:25,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,419 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:25,419 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,419 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:25,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,419 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:25,419 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 19:10:25,419 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:25,419 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 19:10:25,419 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:25,419 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:25,420 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:25,420 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:25,420 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:25,421 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,421 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:25,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:25,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:25,421 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:25,421 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 19:10:25,421 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:25,421 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:25,422 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 19:10:25,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:25,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:25,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:25,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:25,426 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:10:25,426 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-12 19:10:25,426 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:25,426 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:25,426 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,426 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:25,427 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,427 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:25,427 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,427 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:25,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:25,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,427 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:25,427 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:25,428 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:25,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,428 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:25,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:25,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:25,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:25,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:25,428 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:25,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:25,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,429 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:25,431 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,431 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,432 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,435 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,436 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:25,436 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,435 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:25,435 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:25,436 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:25,436 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:25,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:25,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:25,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-12 19:10:25,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:25,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:25,436 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:25,436 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:25,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:25,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:25,436 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:25,437 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:25,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:25,437 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:25,437 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,437 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:25,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:25,438 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:25,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:25,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:25,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:25,438 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,438 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:25,438 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,438 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,438 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 19:10:25,538 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:25,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:25,565 DEBUG [Listener at localhost.localdomain/40405] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,571 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:25,572 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:25,572 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:25,572 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:25,580 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=da5689e107dc59e64b9df0b3d4d1e069, UNASSIGN}] 2023-07-12 19:10:25,581 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=da5689e107dc59e64b9df0b3d4d1e069, UNASSIGN 2023-07-12 19:10:25,582 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=da5689e107dc59e64b9df0b3d4d1e069, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:25,583 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1689189025582"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189025582"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189025582"}]},"ts":"1689189025582"} 2023-07-12 19:10:25,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure da5689e107dc59e64b9df0b3d4d1e069, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:25,750 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,750 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing da5689e107dc59e64b9df0b3d4d1e069, disabling compactions & flushes 2023-07-12 19:10:25,750 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:25,750 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:25,750 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. after waiting 0 ms 2023-07-12 19:10:25,750 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:25,750 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing da5689e107dc59e64b9df0b3d4d1e069 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-12 19:10:25,770 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf1/63e2c719b53846c9b78cfd2bd5b83249 2023-07-12 19:10:25,793 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf2/84b42d9cac7e4afead264d628c31029c 2023-07-12 19:10:25,800 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf1/63e2c719b53846c9b78cfd2bd5b83249 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/63e2c719b53846c9b78cfd2bd5b83249 2023-07-12 19:10:25,807 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/63e2c719b53846c9b78cfd2bd5b83249, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 19:10:25,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.tmp/cf2/84b42d9cac7e4afead264d628c31029c as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/84b42d9cac7e4afead264d628c31029c 2023-07-12 19:10:25,818 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/84b42d9cac7e4afead264d628c31029c, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 19:10:25,819 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for da5689e107dc59e64b9df0b3d4d1e069 in 69ms, sequenceid=82, compaction requested=true 2023-07-12 19:10:25,845 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-12 19:10:25,846 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:25,847 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. 2023-07-12 19:10:25,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for da5689e107dc59e64b9df0b3d4d1e069: 2023-07-12 19:10:25,850 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,851 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=da5689e107dc59e64b9df0b3d4d1e069, regionState=CLOSED 2023-07-12 19:10:25,852 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1689189025850"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189025850"}]},"ts":"1689189025850"} 2023-07-12 19:10:25,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-12 19:10:25,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure da5689e107dc59e64b9df0b3d4d1e069, server=jenkins-hbase20.apache.org,38353,1689189020667 in 269 msec 2023-07-12 19:10:25,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-12 19:10:25,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=da5689e107dc59e64b9df0b3d4d1e069, UNASSIGN in 277 msec 2023-07-12 19:10:25,877 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=da5689e107dc59e64b9df0b3d4d1e069, threads=14 2023-07-12 19:10:25,878 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/0095784930694e8ea8d0f7fbfed3b824 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,878 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/056ee9812a9647dfb4097f307dc577ef for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,878 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/18507120a48446ce918f7936d5c13a7a for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,879 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/1e1bf654215d40e89fdfdf5ad54ab6f0 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,879 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/77afc58b907740b695d7177f54ef9aa6 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,879 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/84b42d9cac7e4afead264d628c31029c for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,879 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/db767ea0c7b04a1296ade791afaab7c7 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,879 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/13441224975444eb9fa642ab4cc24060 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,880 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/1827d2d3fc89457da9bf965aef32a83d for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,884 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/5f51d0d28a1943e5941215a0f0378418 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,884 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/63e2c719b53846c9b78cfd2bd5b83249 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,885 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/650629c852ad4efeaf97a2e3f9684be3 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,885 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/acc7023d5f734655b815d4db916bc879 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,885 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/b69e5cab366442498cae15b556b89a62 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:25,935 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/1e1bf654215d40e89fdfdf5ad54ab6f0, top=true 2023-07-12 19:10:25,941 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/63e2c719b53846c9b78cfd2bd5b83249, top=true 2023-07-12 19:10:25,947 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/0095784930694e8ea8d0f7fbfed3b824, top=true 2023-07-12 19:10:25,948 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/84b42d9cac7e4afead264d628c31029c, top=true 2023-07-12 19:10:25,983 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/13441224975444eb9fa642ab4cc24060, top=true 2023-07-12 19:10:25,984 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/5f51d0d28a1943e5941215a0f0378418, top=true 2023-07-12 19:10:26,021 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.splits/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-84b42d9cac7e4afead264d628c31029c for child: 4fb0a096d88532e6683c4be379845925, parent: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,021 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.splits/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-63e2c719b53846c9b78cfd2bd5b83249 for child: 4fb0a096d88532e6683c4be379845925, parent: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,021 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/84b42d9cac7e4afead264d628c31029c for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,021 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/63e2c719b53846c9b78cfd2bd5b83249 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,035 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.splits/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-0095784930694e8ea8d0f7fbfed3b824 for child: 4fb0a096d88532e6683c4be379845925, parent: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,035 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/0095784930694e8ea8d0f7fbfed3b824 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,036 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.splits/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-1e1bf654215d40e89fdfdf5ad54ab6f0 for child: 4fb0a096d88532e6683c4be379845925, parent: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,038 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/1e1bf654215d40e89fdfdf5ad54ab6f0 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,047 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.splits/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-13441224975444eb9fa642ab4cc24060 for child: 4fb0a096d88532e6683c4be379845925, parent: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,051 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/13441224975444eb9fa642ab4cc24060 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,051 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/.splits/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-5f51d0d28a1943e5941215a0f0378418 for child: 4fb0a096d88532e6683c4be379845925, parent: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,052 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/5f51d0d28a1943e5941215a0f0378418 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,068 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/77afc58b907740b695d7177f54ef9aa6 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,074 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/650629c852ad4efeaf97a2e3f9684be3 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,074 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/18507120a48446ce918f7936d5c13a7a for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,076 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/056ee9812a9647dfb4097f307dc577ef for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,076 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/b69e5cab366442498cae15b556b89a62 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,091 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/1827d2d3fc89457da9bf965aef32a83d for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,091 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/acc7023d5f734655b815d4db916bc879 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,431 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/db767ea0c7b04a1296ade791afaab7c7 for region: da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:26,431 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region da5689e107dc59e64b9df0b3d4d1e069 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-07-12 19:10:26,480 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 19:10:26,482 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 19:10:26,485 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1689189026485"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1689189026485"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1689189026485"}]},"ts":"1689189026485"} 2023-07-12 19:10:26,485 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689189026485"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189026485"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189026485"}]},"ts":"1689189026485"} 2023-07-12 19:10:26,485 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689189026485"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189026485"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189026485"}]},"ts":"1689189026485"} 2023-07-12 19:10:26,527 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cae4263847f309e5972cfdde12d5c6ab, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4fb0a096d88532e6683c4be379845925, ASSIGN}] 2023-07-12 19:10:26,529 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4fb0a096d88532e6683c4be379845925, ASSIGN 2023-07-12 19:10:26,530 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4fb0a096d88532e6683c4be379845925, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,38353,1689189020667; forceNewPlan=false, retain=false 2023-07-12 19:10:26,531 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cae4263847f309e5972cfdde12d5c6ab, ASSIGN 2023-07-12 19:10:26,532 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cae4263847f309e5972cfdde12d5c6ab, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,38353,1689189020667; forceNewPlan=false, retain=false 2023-07-12 19:10:26,680 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-12 19:10:26,682 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=cae4263847f309e5972cfdde12d5c6ab, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:26,682 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689189026682"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189026682"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189026682"}]},"ts":"1689189026682"} 2023-07-12 19:10:26,685 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure cae4263847f309e5972cfdde12d5c6ab, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:26,689 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=4fb0a096d88532e6683c4be379845925, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:26,689 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689189026689"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189026689"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189026689"}]},"ts":"1689189026689"} 2023-07-12 19:10:26,691 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 4fb0a096d88532e6683c4be379845925, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:26,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925. 2023-07-12 19:10:26,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4fb0a096d88532e6683c4be379845925, NAME => 'testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.', STARTKEY => '26', ENDKEY => ''} 2023-07-12 19:10:26,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:26,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:26,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:26,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:26,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:26,845 INFO [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:26,846 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1 2023-07-12 19:10:26,847 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1 2023-07-12 19:10:26,847 INFO [StoreOpener-4fb0a096d88532e6683c4be379845925-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fb0a096d88532e6683c4be379845925 columnFamilyName cf1 2023-07-12 19:10:26,879 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/1827d2d3fc89457da9bf965aef32a83d.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/1827d2d3fc89457da9bf965aef32a83d-top 2023-07-12 19:10:26,888 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/650629c852ad4efeaf97a2e3f9684be3.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/650629c852ad4efeaf97a2e3f9684be3-top 2023-07-12 19:10:26,895 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/acc7023d5f734655b815d4db916bc879.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/acc7023d5f734655b815d4db916bc879-top 2023-07-12 19:10:26,903 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/b69e5cab366442498cae15b556b89a62.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/b69e5cab366442498cae15b556b89a62-top 2023-07-12 19:10:26,909 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-13441224975444eb9fa642ab4cc24060 2023-07-12 19:10:26,915 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-5f51d0d28a1943e5941215a0f0378418 2023-07-12 19:10:26,922 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-63e2c719b53846c9b78cfd2bd5b83249 2023-07-12 19:10:26,923 INFO [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(310): Store=4fb0a096d88532e6683c4be379845925/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:26,923 INFO [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:26,924 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2 2023-07-12 19:10:26,924 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2 2023-07-12 19:10:26,925 INFO [StoreOpener-4fb0a096d88532e6683c4be379845925-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fb0a096d88532e6683c4be379845925 columnFamilyName cf2 2023-07-12 19:10:26,941 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/056ee9812a9647dfb4097f307dc577ef.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/056ee9812a9647dfb4097f307dc577ef-top 2023-07-12 19:10:26,947 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/18507120a48446ce918f7936d5c13a7a.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/18507120a48446ce918f7936d5c13a7a-top 2023-07-12 19:10:26,953 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/77afc58b907740b695d7177f54ef9aa6.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/77afc58b907740b695d7177f54ef9aa6-top 2023-07-12 19:10:26,960 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/db767ea0c7b04a1296ade791afaab7c7.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/db767ea0c7b04a1296ade791afaab7c7-top 2023-07-12 19:10:26,966 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-0095784930694e8ea8d0f7fbfed3b824 2023-07-12 19:10:26,972 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-1e1bf654215d40e89fdfdf5ad54ab6f0 2023-07-12 19:10:26,979 DEBUG [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-84b42d9cac7e4afead264d628c31029c 2023-07-12 19:10:26,979 INFO [StoreOpener-4fb0a096d88532e6683c4be379845925-1] regionserver.HStore(310): Store=4fb0a096d88532e6683c4be379845925/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:26,980 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:26,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:26,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:10:26,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:26,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 4fb0a096d88532e6683c4be379845925; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70655047, jitterRate=0.05284224450588226}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:10:26,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 4fb0a096d88532e6683c4be379845925: 2023-07-12 19:10:26,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925., pid=18, masterSystemTime=1689189026837 2023-07-12 19:10:26,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:26,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:10:26,992 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:10:26,992 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:10:26,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925. 2023-07-12 19:10:26,994 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925. 2023-07-12 19:10:26,994 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925. 2023-07-12 19:10:26,994 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1912): 4fb0a096d88532e6683c4be379845925/cf2 is initiating minor compaction (all files) 2023-07-12 19:10:26,994 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1912): 4fb0a096d88532e6683c4be379845925/cf1 is initiating minor compaction (all files) 2023-07-12 19:10:26,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925. 2023-07-12 19:10:26,995 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 4fb0a096d88532e6683c4be379845925/cf2 in testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925. 2023-07-12 19:10:26,995 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 4fb0a096d88532e6683c4be379845925/cf1 in testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925. 2023-07-12 19:10:26,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab. 2023-07-12 19:10:26,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cae4263847f309e5972cfdde12d5c6ab, NAME => 'testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.', STARTKEY => '', ENDKEY => '26'} 2023-07-12 19:10:26,995 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=4fb0a096d88532e6683c4be379845925, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:26,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:26,995 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/b69e5cab366442498cae15b556b89a62.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/b69e5cab366442498cae15b556b89a62-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/1827d2d3fc89457da9bf965aef32a83d.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/1827d2d3fc89457da9bf965aef32a83d-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/acc7023d5f734655b815d4db916bc879.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/acc7023d5f734655b815d4db916bc879-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/650629c852ad4efeaf97a2e3f9684be3.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/650629c852ad4efeaf97a2e3f9684be3-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-13441224975444eb9fa642ab4cc24060, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-5f51d0d28a1943e5941215a0f0378418, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-63e2c719b53846c9b78cfd2bd5b83249] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/.tmp, totalSize=34.9 K 2023-07-12 19:10:26,995 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/77afc58b907740b695d7177f54ef9aa6.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/77afc58b907740b695d7177f54ef9aa6-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/18507120a48446ce918f7936d5c13a7a.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/18507120a48446ce918f7936d5c13a7a-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/db767ea0c7b04a1296ade791afaab7c7.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/db767ea0c7b04a1296ade791afaab7c7-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/056ee9812a9647dfb4097f307dc577ef.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/056ee9812a9647dfb4097f307dc577ef-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-0095784930694e8ea8d0f7fbfed3b824, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-1e1bf654215d40e89fdfdf5ad54ab6f0, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-84b42d9cac7e4afead264d628c31029c] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/.tmp, totalSize=34.9 K 2023-07-12 19:10:26,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter cae4263847f309e5972cfdde12d5c6ab 2023-07-12 19:10:26,995 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689189026995"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189026995"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189026995"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189026995"}]},"ts":"1689189026995"} 2023-07-12 19:10:26,997 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:26,997 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for cae4263847f309e5972cfdde12d5c6ab 2023-07-12 19:10:26,997 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for cae4263847f309e5972cfdde12d5c6ab 2023-07-12 19:10:26,999 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 77afc58b907740b695d7177f54ef9aa6.da5689e107dc59e64b9df0b3d4d1e069, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689189023913 2023-07-12 19:10:27,000 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting b69e5cab366442498cae15b556b89a62.da5689e107dc59e64b9df0b3d4d1e069, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689189023913 2023-07-12 19:10:27,001 INFO [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region cae4263847f309e5972cfdde12d5c6ab 2023-07-12 19:10:27,001 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 1827d2d3fc89457da9bf965aef32a83d.da5689e107dc59e64b9df0b3d4d1e069, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689189024278 2023-07-12 19:10:27,001 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 18507120a48446ce918f7936d5c13a7a.da5689e107dc59e64b9df0b3d4d1e069, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689189024278 2023-07-12 19:10:27,001 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting db767ea0c7b04a1296ade791afaab7c7.da5689e107dc59e64b9df0b3d4d1e069, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689189024540 2023-07-12 19:10:27,001 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting acc7023d5f734655b815d4db916bc879.da5689e107dc59e64b9df0b3d4d1e069, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689189024540 2023-07-12 19:10:27,002 DEBUG [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1 2023-07-12 19:10:27,002 DEBUG [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1 2023-07-12 19:10:27,002 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 650629c852ad4efeaf97a2e3f9684be3.da5689e107dc59e64b9df0b3d4d1e069, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689189024814 2023-07-12 19:10:27,002 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 056ee9812a9647dfb4097f307dc577ef.da5689e107dc59e64b9df0b3d4d1e069, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689189024814 2023-07-12 19:10:27,003 INFO [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cae4263847f309e5972cfdde12d5c6ab columnFamilyName cf1 2023-07-12 19:10:27,003 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-13441224975444eb9fa642ab4cc24060, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689189025051 2023-07-12 19:10:27,004 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-0095784930694e8ea8d0f7fbfed3b824, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689189025051 2023-07-12 19:10:27,005 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-5f51d0d28a1943e5941215a0f0378418, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689189025303 2023-07-12 19:10:27,005 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-1e1bf654215d40e89fdfdf5ad54ab6f0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689189025303 2023-07-12 19:10:27,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-07-12 19:10:27,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 4fb0a096d88532e6683c4be379845925, server=jenkins-hbase20.apache.org,38353,1689189020667 in 310 msec 2023-07-12 19:10:27,006 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-84b42d9cac7e4afead264d628c31029c, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689189025543 2023-07-12 19:10:27,007 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-63e2c719b53846c9b78cfd2bd5b83249, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689189025543 2023-07-12 19:10:27,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4fb0a096d88532e6683c4be379845925, ASSIGN in 479 msec 2023-07-12 19:10:27,022 DEBUG [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/1827d2d3fc89457da9bf965aef32a83d.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/1827d2d3fc89457da9bf965aef32a83d-bottom 2023-07-12 19:10:27,028 DEBUG [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/650629c852ad4efeaf97a2e3f9684be3.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/650629c852ad4efeaf97a2e3f9684be3-bottom 2023-07-12 19:10:27,042 DEBUG [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/acc7023d5f734655b815d4db916bc879.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/acc7023d5f734655b815d4db916bc879-bottom 2023-07-12 19:10:27,042 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] throttle.PressureAwareThroughputController(145): 4fb0a096d88532e6683c4be379845925#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:27,047 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4fb0a096d88532e6683c4be379845925#cf1#compaction#15 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:27,050 DEBUG [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/b69e5cab366442498cae15b556b89a62.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/b69e5cab366442498cae15b556b89a62-bottom 2023-07-12 19:10:27,050 INFO [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] regionserver.HStore(310): Store=cae4263847f309e5972cfdde12d5c6ab/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:27,050 INFO [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region cae4263847f309e5972cfdde12d5c6ab 2023-07-12 19:10:27,051 DEBUG [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2 2023-07-12 19:10:27,051 DEBUG [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2 2023-07-12 19:10:27,052 INFO [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cae4263847f309e5972cfdde12d5c6ab columnFamilyName cf2 2023-07-12 19:10:27,064 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/.tmp/cf2/3686e9d06aa84bcb9c65c578ba75cf2e as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/3686e9d06aa84bcb9c65c578ba75cf2e 2023-07-12 19:10:27,069 DEBUG [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/056ee9812a9647dfb4097f307dc577ef.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/056ee9812a9647dfb4097f307dc577ef-bottom 2023-07-12 19:10:27,073 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/.tmp/cf1/52982cdaa4df4a5eb932e3f7defbc310 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/52982cdaa4df4a5eb932e3f7defbc310 2023-07-12 19:10:27,077 DEBUG [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/18507120a48446ce918f7936d5c13a7a.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/18507120a48446ce918f7936d5c13a7a-bottom 2023-07-12 19:10:27,082 DEBUG [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/77afc58b907740b695d7177f54ef9aa6.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/77afc58b907740b695d7177f54ef9aa6-bottom 2023-07-12 19:10:27,087 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4fb0a096d88532e6683c4be379845925/cf2 of 4fb0a096d88532e6683c4be379845925 into 3686e9d06aa84bcb9c65c578ba75cf2e(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:27,087 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4fb0a096d88532e6683c4be379845925/cf1 of 4fb0a096d88532e6683c4be379845925 into 52982cdaa4df4a5eb932e3f7defbc310(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:27,087 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4fb0a096d88532e6683c4be379845925: 2023-07-12 19:10:27,087 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4fb0a096d88532e6683c4be379845925: 2023-07-12 19:10:27,087 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925., storeName=4fb0a096d88532e6683c4be379845925/cf2, priority=9, startTime=1689189026992; duration=0sec 2023-07-12 19:10:27,087 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925., storeName=4fb0a096d88532e6683c4be379845925/cf1, priority=9, startTime=1689189026990; duration=0sec 2023-07-12 19:10:27,088 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:27,088 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:27,088 DEBUG [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/db767ea0c7b04a1296ade791afaab7c7.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/db767ea0c7b04a1296ade791afaab7c7-bottom 2023-07-12 19:10:27,088 INFO [StoreOpener-cae4263847f309e5972cfdde12d5c6ab-1] regionserver.HStore(310): Store=cae4263847f309e5972cfdde12d5c6ab/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:27,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab 2023-07-12 19:10:27,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab 2023-07-12 19:10:27,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:10:27,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for cae4263847f309e5972cfdde12d5c6ab 2023-07-12 19:10:27,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened cae4263847f309e5972cfdde12d5c6ab; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73563814, jitterRate=0.09618625044822693}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:10:27,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for cae4263847f309e5972cfdde12d5c6ab: 2023-07-12 19:10:27,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab., pid=17, masterSystemTime=1689189026837 2023-07-12 19:10:27,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:27,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:10:27,096 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-07-12 19:10:27,096 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-07-12 19:10:27,096 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab. 2023-07-12 19:10:27,096 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab. 2023-07-12 19:10:27,097 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1912): cae4263847f309e5972cfdde12d5c6ab/cf1 is initiating minor compaction (all files) 2023-07-12 19:10:27,097 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1912): cae4263847f309e5972cfdde12d5c6ab/cf2 is initiating minor compaction (all files) 2023-07-12 19:10:27,097 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of cae4263847f309e5972cfdde12d5c6ab/cf1 in testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab. 2023-07-12 19:10:27,097 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2259): Starting compaction of cae4263847f309e5972cfdde12d5c6ab/cf2 in testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab. 2023-07-12 19:10:27,097 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/b69e5cab366442498cae15b556b89a62.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/b69e5cab366442498cae15b556b89a62-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/1827d2d3fc89457da9bf965aef32a83d.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/1827d2d3fc89457da9bf965aef32a83d-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/acc7023d5f734655b815d4db916bc879.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/acc7023d5f734655b815d4db916bc879-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/650629c852ad4efeaf97a2e3f9684be3.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/650629c852ad4efeaf97a2e3f9684be3-bottom] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/.tmp, totalSize=19.9 K 2023-07-12 19:10:27,097 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/77afc58b907740b695d7177f54ef9aa6.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/77afc58b907740b695d7177f54ef9aa6-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/18507120a48446ce918f7936d5c13a7a.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/18507120a48446ce918f7936d5c13a7a-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/db767ea0c7b04a1296ade791afaab7c7.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/db767ea0c7b04a1296ade791afaab7c7-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/056ee9812a9647dfb4097f307dc577ef.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/056ee9812a9647dfb4097f307dc577ef-bottom] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/.tmp, totalSize=19.9 K 2023-07-12 19:10:27,098 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting b69e5cab366442498cae15b556b89a62.da5689e107dc59e64b9df0b3d4d1e069, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689189023913 2023-07-12 19:10:27,098 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 77afc58b907740b695d7177f54ef9aa6.da5689e107dc59e64b9df0b3d4d1e069, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689189023913 2023-07-12 19:10:27,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab. 2023-07-12 19:10:27,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab. 2023-07-12 19:10:27,099 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 18507120a48446ce918f7936d5c13a7a.da5689e107dc59e64b9df0b3d4d1e069, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689189024278 2023-07-12 19:10:27,099 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 1827d2d3fc89457da9bf965aef32a83d.da5689e107dc59e64b9df0b3d4d1e069, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689189024278 2023-07-12 19:10:27,100 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=cae4263847f309e5972cfdde12d5c6ab, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:27,100 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689189027099"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189027099"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189027099"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189027099"}]},"ts":"1689189027099"} 2023-07-12 19:10:27,100 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting db767ea0c7b04a1296ade791afaab7c7.da5689e107dc59e64b9df0b3d4d1e069, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689189024540 2023-07-12 19:10:27,100 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting acc7023d5f734655b815d4db916bc879.da5689e107dc59e64b9df0b3d4d1e069, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689189024540 2023-07-12 19:10:27,101 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 056ee9812a9647dfb4097f307dc577ef.da5689e107dc59e64b9df0b3d4d1e069, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689189024814 2023-07-12 19:10:27,101 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 650629c852ad4efeaf97a2e3f9684be3.da5689e107dc59e64b9df0b3d4d1e069, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689189024814 2023-07-12 19:10:27,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-07-12 19:10:27,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure cae4263847f309e5972cfdde12d5c6ab, server=jenkins-hbase20.apache.org,38353,1689189020667 in 418 msec 2023-07-12 19:10:27,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-07-12 19:10:27,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cae4263847f309e5972cfdde12d5c6ab, ASSIGN in 580 msec 2023-07-12 19:10:27,114 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 in 1.5450 sec 2023-07-12 19:10:27,115 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] throttle.PressureAwareThroughputController(145): cae4263847f309e5972cfdde12d5c6ab#cf1#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:27,115 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] throttle.PressureAwareThroughputController(145): cae4263847f309e5972cfdde12d5c6ab#cf2#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:27,128 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/.tmp/cf2/7bf176422827406e84b623ffc0fee010 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/7bf176422827406e84b623ffc0fee010 2023-07-12 19:10:27,134 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in cae4263847f309e5972cfdde12d5c6ab/cf2 of cae4263847f309e5972cfdde12d5c6ab into 7bf176422827406e84b623ffc0fee010(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:27,134 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for cae4263847f309e5972cfdde12d5c6ab: 2023-07-12 19:10:27,134 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab., storeName=cae4263847f309e5972cfdde12d5c6ab/cf2, priority=12, startTime=1689189027096; duration=0sec 2023-07-12 19:10:27,134 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:27,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38353] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 148.251.75.209:59964 deadline: 1689189087328, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069. is not online on jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:27,466 DEBUG [Listener at localhost.localdomain/40405] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-07-12 19:10:27,479 INFO [Listener at localhost.localdomain/40405] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-12 19:10:27,482 DEBUG [Listener at localhost.localdomain/40405] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-07-12 19:10:27,516 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:10:27,525 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-07-12 19:10:27,530 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitTableRegionUnevenDaughter 2023-07-12 19:10:27,531 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/.tmp/cf1/482ab5c2dd6e4ac79315cd8506f04337 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/482ab5c2dd6e4ac79315cd8506f04337 2023-07-12 19:10:27,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:27,540 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in cae4263847f309e5972cfdde12d5c6ab/cf1 of cae4263847f309e5972cfdde12d5c6ab into 482ab5c2dd6e4ac79315cd8506f04337(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:27,540 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for cae4263847f309e5972cfdde12d5c6ab: 2023-07-12 19:10:27,541 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab., storeName=cae4263847f309e5972cfdde12d5c6ab/cf1, priority=12, startTime=1689189027095; duration=0sec 2023-07-12 19:10:27,541 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:27,543 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189027543"}]},"ts":"1689189027543"} 2023-07-12 19:10:27,545 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-07-12 19:10:27,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-07-12 19:10:27,572 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-07-12 19:10:27,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cae4263847f309e5972cfdde12d5c6ab, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4fb0a096d88532e6683c4be379845925, UNASSIGN}] 2023-07-12 19:10:27,575 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4fb0a096d88532e6683c4be379845925, UNASSIGN 2023-07-12 19:10:27,576 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=4fb0a096d88532e6683c4be379845925, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:27,576 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689189027576"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189027576"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189027576"}]},"ts":"1689189027576"} 2023-07-12 19:10:27,578 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 4fb0a096d88532e6683c4be379845925, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:27,581 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cae4263847f309e5972cfdde12d5c6ab, UNASSIGN 2023-07-12 19:10:27,582 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=cae4263847f309e5972cfdde12d5c6ab, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:27,582 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689189027582"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189027582"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189027582"}]},"ts":"1689189027582"} 2023-07-12 19:10:27,584 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure cae4263847f309e5972cfdde12d5c6ab, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:27,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-07-12 19:10:27,731 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close cae4263847f309e5972cfdde12d5c6ab 2023-07-12 19:10:27,731 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing cae4263847f309e5972cfdde12d5c6ab, disabling compactions & flushes 2023-07-12 19:10:27,731 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab. 2023-07-12 19:10:27,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab. 2023-07-12 19:10:27,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab. after waiting 0 ms 2023-07-12 19:10:27,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab. 2023-07-12 19:10:27,737 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/b69e5cab366442498cae15b556b89a62.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/b69e5cab366442498cae15b556b89a62-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/1827d2d3fc89457da9bf965aef32a83d.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/1827d2d3fc89457da9bf965aef32a83d-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/acc7023d5f734655b815d4db916bc879.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/acc7023d5f734655b815d4db916bc879-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/650629c852ad4efeaf97a2e3f9684be3.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/650629c852ad4efeaf97a2e3f9684be3-bottom] to archive 2023-07-12 19:10:27,738 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:10:27,743 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/b69e5cab366442498cae15b556b89a62.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/b69e5cab366442498cae15b556b89a62.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,745 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/1827d2d3fc89457da9bf965aef32a83d.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/1827d2d3fc89457da9bf965aef32a83d.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,746 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/acc7023d5f734655b815d4db916bc879.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/acc7023d5f734655b815d4db916bc879.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,748 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/650629c852ad4efeaf97a2e3f9684be3.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/650629c852ad4efeaf97a2e3f9684be3.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,770 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/77afc58b907740b695d7177f54ef9aa6.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/77afc58b907740b695d7177f54ef9aa6-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/18507120a48446ce918f7936d5c13a7a.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/18507120a48446ce918f7936d5c13a7a-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/db767ea0c7b04a1296ade791afaab7c7.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/db767ea0c7b04a1296ade791afaab7c7-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/056ee9812a9647dfb4097f307dc577ef.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/056ee9812a9647dfb4097f307dc577ef-bottom] to archive 2023-07-12 19:10:27,771 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:10:27,774 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/77afc58b907740b695d7177f54ef9aa6.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/77afc58b907740b695d7177f54ef9aa6.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,776 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/18507120a48446ce918f7936d5c13a7a.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/18507120a48446ce918f7936d5c13a7a.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,778 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/db767ea0c7b04a1296ade791afaab7c7.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/db767ea0c7b04a1296ade791afaab7c7.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,779 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/056ee9812a9647dfb4097f307dc577ef.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/056ee9812a9647dfb4097f307dc577ef.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 19:10:27,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:27,785 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab. 2023-07-12 19:10:27,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for cae4263847f309e5972cfdde12d5c6ab: 2023-07-12 19:10:27,787 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed cae4263847f309e5972cfdde12d5c6ab 2023-07-12 19:10:27,787 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:27,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 4fb0a096d88532e6683c4be379845925, disabling compactions & flushes 2023-07-12 19:10:27,787 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925. 2023-07-12 19:10:27,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925. 2023-07-12 19:10:27,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925. after waiting 0 ms 2023-07-12 19:10:27,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925. 2023-07-12 19:10:27,788 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=cae4263847f309e5972cfdde12d5c6ab, regionState=CLOSED 2023-07-12 19:10:27,788 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689189027788"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189027788"}]},"ts":"1689189027788"} 2023-07-12 19:10:27,789 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/b69e5cab366442498cae15b556b89a62.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/b69e5cab366442498cae15b556b89a62-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/1827d2d3fc89457da9bf965aef32a83d.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/1827d2d3fc89457da9bf965aef32a83d-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/acc7023d5f734655b815d4db916bc879.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/acc7023d5f734655b815d4db916bc879-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/650629c852ad4efeaf97a2e3f9684be3.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/650629c852ad4efeaf97a2e3f9684be3-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-13441224975444eb9fa642ab4cc24060, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-5f51d0d28a1943e5941215a0f0378418, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-63e2c719b53846c9b78cfd2bd5b83249] to archive 2023-07-12 19:10:27,790 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:10:27,793 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-07-12 19:10:27,793 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/b69e5cab366442498cae15b556b89a62.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/b69e5cab366442498cae15b556b89a62.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,793 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure cae4263847f309e5972cfdde12d5c6ab, server=jenkins-hbase20.apache.org,38353,1689189020667 in 206 msec 2023-07-12 19:10:27,795 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/1827d2d3fc89457da9bf965aef32a83d.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/1827d2d3fc89457da9bf965aef32a83d.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cae4263847f309e5972cfdde12d5c6ab, UNASSIGN in 220 msec 2023-07-12 19:10:27,797 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/acc7023d5f734655b815d4db916bc879.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/acc7023d5f734655b815d4db916bc879.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,799 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/650629c852ad4efeaf97a2e3f9684be3.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/650629c852ad4efeaf97a2e3f9684be3.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,800 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-13441224975444eb9fa642ab4cc24060 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-13441224975444eb9fa642ab4cc24060 2023-07-12 19:10:27,802 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-5f51d0d28a1943e5941215a0f0378418 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-5f51d0d28a1943e5941215a0f0378418 2023-07-12 19:10:27,803 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-63e2c719b53846c9b78cfd2bd5b83249 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-63e2c719b53846c9b78cfd2bd5b83249 2023-07-12 19:10:27,805 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/77afc58b907740b695d7177f54ef9aa6.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/77afc58b907740b695d7177f54ef9aa6-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/18507120a48446ce918f7936d5c13a7a.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/18507120a48446ce918f7936d5c13a7a-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/db767ea0c7b04a1296ade791afaab7c7.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/db767ea0c7b04a1296ade791afaab7c7-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/056ee9812a9647dfb4097f307dc577ef.da5689e107dc59e64b9df0b3d4d1e069->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/056ee9812a9647dfb4097f307dc577ef-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-0095784930694e8ea8d0f7fbfed3b824, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-1e1bf654215d40e89fdfdf5ad54ab6f0, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-84b42d9cac7e4afead264d628c31029c] to archive 2023-07-12 19:10:27,806 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:10:27,809 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/77afc58b907740b695d7177f54ef9aa6.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/77afc58b907740b695d7177f54ef9aa6.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,810 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/18507120a48446ce918f7936d5c13a7a.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/18507120a48446ce918f7936d5c13a7a.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,812 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/db767ea0c7b04a1296ade791afaab7c7.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/db767ea0c7b04a1296ade791afaab7c7.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,814 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/056ee9812a9647dfb4097f307dc577ef.da5689e107dc59e64b9df0b3d4d1e069 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/056ee9812a9647dfb4097f307dc577ef.da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:27,816 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-0095784930694e8ea8d0f7fbfed3b824 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-0095784930694e8ea8d0f7fbfed3b824 2023-07-12 19:10:27,817 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-1e1bf654215d40e89fdfdf5ad54ab6f0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-1e1bf654215d40e89fdfdf5ad54ab6f0 2023-07-12 19:10:27,819 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-84b42d9cac7e4afead264d628c31029c to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/testSplitTableRegionUnevenDaughter=da5689e107dc59e64b9df0b3d4d1e069-84b42d9cac7e4afead264d628c31029c 2023-07-12 19:10:27,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 19:10:27,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:27,825 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925. 2023-07-12 19:10:27,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 4fb0a096d88532e6683c4be379845925: 2023-07-12 19:10:27,828 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:27,829 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=4fb0a096d88532e6683c4be379845925, regionState=CLOSED 2023-07-12 19:10:27,829 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689189027829"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189027829"}]},"ts":"1689189027829"} 2023-07-12 19:10:27,831 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 19:10:27,835 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-07-12 19:10:27,835 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 4fb0a096d88532e6683c4be379845925, server=jenkins-hbase20.apache.org,38353,1689189020667 in 255 msec 2023-07-12 19:10:27,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-07-12 19:10:27,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4fb0a096d88532e6683c4be379845925, UNASSIGN in 261 msec 2023-07-12 19:10:27,839 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189027839"}]},"ts":"1689189027839"} 2023-07-12 19:10:27,841 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-07-12 19:10:27,844 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-07-12 19:10:27,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-07-12 19:10:27,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 325 msec 2023-07-12 19:10:27,908 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-12 19:10:27,909 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-12 19:10:28,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-07-12 19:10:28,161 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-07-12 19:10:28,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitTableRegionUnevenDaughter 2023-07-12 19:10:28,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:28,181 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:28,183 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:28,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-12 19:10:28,193 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:28,193 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab 2023-07-12 19:10:28,194 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:28,199 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/recovered.edits] 2023-07-12 19:10:28,199 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/recovered.edits] 2023-07-12 19:10:28,199 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/recovered.edits] 2023-07-12 19:10:28,205 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/52982cdaa4df4a5eb932e3f7defbc310 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf1/52982cdaa4df4a5eb932e3f7defbc310 2023-07-12 19:10:28,205 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/482ab5c2dd6e4ac79315cd8506f04337 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf1/482ab5c2dd6e4ac79315cd8506f04337 2023-07-12 19:10:28,205 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/13441224975444eb9fa642ab4cc24060 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/13441224975444eb9fa642ab4cc24060 2023-07-12 19:10:28,207 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/1827d2d3fc89457da9bf965aef32a83d to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/1827d2d3fc89457da9bf965aef32a83d 2023-07-12 19:10:28,208 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/3686e9d06aa84bcb9c65c578ba75cf2e to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/cf2/3686e9d06aa84bcb9c65c578ba75cf2e 2023-07-12 19:10:28,208 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/7bf176422827406e84b623ffc0fee010 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/cf2/7bf176422827406e84b623ffc0fee010 2023-07-12 19:10:28,209 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/5f51d0d28a1943e5941215a0f0378418 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/5f51d0d28a1943e5941215a0f0378418 2023-07-12 19:10:28,212 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/63e2c719b53846c9b78cfd2bd5b83249 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/63e2c719b53846c9b78cfd2bd5b83249 2023-07-12 19:10:28,214 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/recovered.edits/90.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925/recovered.edits/90.seqid 2023-07-12 19:10:28,214 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/650629c852ad4efeaf97a2e3f9684be3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/650629c852ad4efeaf97a2e3f9684be3 2023-07-12 19:10:28,215 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/recovered.edits/90.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab/recovered.edits/90.seqid 2023-07-12 19:10:28,216 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:28,217 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/cae4263847f309e5972cfdde12d5c6ab 2023-07-12 19:10:28,217 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/acc7023d5f734655b815d4db916bc879 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/acc7023d5f734655b815d4db916bc879 2023-07-12 19:10:28,219 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/b69e5cab366442498cae15b556b89a62 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf1/b69e5cab366442498cae15b556b89a62 2023-07-12 19:10:28,222 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/0095784930694e8ea8d0f7fbfed3b824 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/0095784930694e8ea8d0f7fbfed3b824 2023-07-12 19:10:28,225 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/056ee9812a9647dfb4097f307dc577ef to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/056ee9812a9647dfb4097f307dc577ef 2023-07-12 19:10:28,226 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/18507120a48446ce918f7936d5c13a7a to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/18507120a48446ce918f7936d5c13a7a 2023-07-12 19:10:28,228 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/1e1bf654215d40e89fdfdf5ad54ab6f0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/1e1bf654215d40e89fdfdf5ad54ab6f0 2023-07-12 19:10:28,229 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/77afc58b907740b695d7177f54ef9aa6 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/77afc58b907740b695d7177f54ef9aa6 2023-07-12 19:10:28,231 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/84b42d9cac7e4afead264d628c31029c to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/84b42d9cac7e4afead264d628c31029c 2023-07-12 19:10:28,233 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/db767ea0c7b04a1296ade791afaab7c7 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/cf2/db767ea0c7b04a1296ade791afaab7c7 2023-07-12 19:10:28,236 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/recovered.edits/85.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069/recovered.edits/85.seqid 2023-07-12 19:10:28,237 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionUnevenDaughter/da5689e107dc59e64b9df0b3d4d1e069 2023-07-12 19:10:28,237 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-07-12 19:10:28,239 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:28,259 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-07-12 19:10:28,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-12 19:10:28,300 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-07-12 19:10:28,302 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:28,302 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-07-12 19:10:28,302 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189028302"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:28,302 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189028302"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:28,302 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189028302"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:28,305 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-12 19:10:28,305 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => da5689e107dc59e64b9df0b3d4d1e069, NAME => 'testSplitTableRegionUnevenDaughter,,1689189023403.da5689e107dc59e64b9df0b3d4d1e069.', STARTKEY => '', ENDKEY => ''}, {ENCODED => cae4263847f309e5972cfdde12d5c6ab, NAME => 'testSplitTableRegionUnevenDaughter,,1689189025566.cae4263847f309e5972cfdde12d5c6ab.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 4fb0a096d88532e6683c4be379845925, NAME => 'testSplitTableRegionUnevenDaughter,26,1689189025566.4fb0a096d88532e6683c4be379845925.', STARTKEY => '26', ENDKEY => ''}] 2023-07-12 19:10:28,305 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-07-12 19:10:28,306 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689189028305"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:28,308 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-07-12 19:10:28,310 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:28,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 137 msec 2023-07-12 19:10:28,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-12 19:10:28,490 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-07-12 19:10:28,517 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=393 (was 361) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:43282 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:51540 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:51866 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:47514 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:51606 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:51374 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase20:38353-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7acefbf8-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7acefbf8-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7acefbf8-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:43378 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:47416 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:43214 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:43330 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:51568 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:47532 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:47520 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:51574 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7acefbf8-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:47756 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7acefbf8-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:43392 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=839 (was 698) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=231 (was 242), ProcessCount=174 (was 181), AvailableMemoryMB=5992 (was 6087) 2023-07-12 19:10:28,530 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=393, OpenFileDescriptor=839, MaxFileDescriptor=60000, SystemLoadAverage=231, ProcessCount=174, AvailableMemoryMB=5990 2023-07-12 19:10:28,531 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:10:28,535 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 19:10:28,536 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 19:10:28,538 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:28,540 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 19:10:28,541 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 19:10:28,543 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:28,543 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8 empty. 2023-07-12 19:10:28,544 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:28,544 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-07-12 19:10:28,563 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-07-12 19:10:28,564 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7675): creating {ENCODED => d0505bae67c35c958e3be134081a0fc8, NAME => 'testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp 2023-07-12 19:10:28,578 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:28,579 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1604): Closing d0505bae67c35c958e3be134081a0fc8, disabling compactions & flushes 2023-07-12 19:10:28,579 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:28,579 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:28,579 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. after waiting 0 ms 2023-07-12 19:10:28,579 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:28,579 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:28,579 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1558): Region close journal for d0505bae67c35c958e3be134081a0fc8: 2023-07-12 19:10:28,581 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 19:10:28,583 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689189028582"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189028582"}]},"ts":"1689189028582"} 2023-07-12 19:10:28,584 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 19:10:28,585 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 19:10:28,586 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189028586"}]},"ts":"1689189028586"} 2023-07-12 19:10:28,587 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-07-12 19:10:28,589 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 19:10:28,591 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 19:10:28,591 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 19:10:28,591 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 19:10:28,591 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 19:10:28,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, ASSIGN}] 2023-07-12 19:10:28,593 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, ASSIGN 2023-07-12 19:10:28,594 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,38353,1689189020667; forceNewPlan=false, retain=false 2023-07-12 19:10:28,744 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 19:10:28,747 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=d0505bae67c35c958e3be134081a0fc8, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:28,747 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689189028746"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189028746"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189028746"}]},"ts":"1689189028746"} 2023-07-12 19:10:28,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:28,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:28,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d0505bae67c35c958e3be134081a0fc8, NAME => 'testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8.', STARTKEY => '', ENDKEY => ''} 2023-07-12 19:10:28,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:28,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:28,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:28,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:28,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:28,921 INFO [StoreOpener-d0505bae67c35c958e3be134081a0fc8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:28,923 DEBUG [StoreOpener-d0505bae67c35c958e3be134081a0fc8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1 2023-07-12 19:10:28,923 DEBUG [StoreOpener-d0505bae67c35c958e3be134081a0fc8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1 2023-07-12 19:10:28,923 INFO [StoreOpener-d0505bae67c35c958e3be134081a0fc8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d0505bae67c35c958e3be134081a0fc8 columnFamilyName cf1 2023-07-12 19:10:28,924 INFO [StoreOpener-d0505bae67c35c958e3be134081a0fc8-1] regionserver.HStore(310): Store=d0505bae67c35c958e3be134081a0fc8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:28,924 INFO [StoreOpener-d0505bae67c35c958e3be134081a0fc8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:28,926 DEBUG [StoreOpener-d0505bae67c35c958e3be134081a0fc8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2 2023-07-12 19:10:28,926 DEBUG [StoreOpener-d0505bae67c35c958e3be134081a0fc8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2 2023-07-12 19:10:28,927 INFO [StoreOpener-d0505bae67c35c958e3be134081a0fc8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d0505bae67c35c958e3be134081a0fc8 columnFamilyName cf2 2023-07-12 19:10:28,928 INFO [StoreOpener-d0505bae67c35c958e3be134081a0fc8-1] regionserver.HStore(310): Store=d0505bae67c35c958e3be134081a0fc8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:28,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:28,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:28,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:10:28,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:28,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 19:10:28,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened d0505bae67c35c958e3be134081a0fc8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64307533, jitterRate=-0.04174308478832245}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:10:28,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for d0505bae67c35c958e3be134081a0fc8: 2023-07-12 19:10:28,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8., pid=27, masterSystemTime=1689189028905 2023-07-12 19:10:28,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:28,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:28,939 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=d0505bae67c35c958e3be134081a0fc8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:28,939 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689189028939"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189028939"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189028939"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189028939"}]},"ts":"1689189028939"} 2023-07-12 19:10:28,943 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-07-12 19:10:28,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 in 189 msec 2023-07-12 19:10:28,946 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-07-12 19:10:28,946 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, ASSIGN in 353 msec 2023-07-12 19:10:28,947 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 19:10:28,948 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189028947"}]},"ts":"1689189028947"} 2023-07-12 19:10:28,949 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-07-12 19:10:28,951 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 19:10:28,953 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 414 msec 2023-07-12 19:10:29,056 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:29,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-12 19:10:29,059 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-12 19:10:29,059 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,060 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-12 19:10:29,060 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-12 19:10:29,061 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,061 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,062 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,062 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,062 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,062 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,063 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,063 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:29,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,063 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,064 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,064 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:29,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:29,064 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:29,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:29,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,064 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:29,064 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:29,064 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:29,065 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:29,065 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,065 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:29,065 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:29,067 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:29,067 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:29,067 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:29,067 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:29,067 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,067 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:29,068 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:29,068 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:29,068 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:29,068 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,069 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:29,069 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:29,069 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:29,069 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:29,069 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. started... 2023-07-12 19:10:29,070 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0505bae67c35c958e3be134081a0fc8 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 19:10:29,077 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,078 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,078 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,078 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:29,078 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,078 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:29,083 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf1/5aef80d484ce4cd6b6fc892fa133a77b 2023-07-12 19:10:29,097 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf2/58e66d0dabaf481cb438e9ead062860d 2023-07-12 19:10:29,102 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf1/5aef80d484ce4cd6b6fc892fa133a77b as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5aef80d484ce4cd6b6fc892fa133a77b 2023-07-12 19:10:29,107 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5aef80d484ce4cd6b6fc892fa133a77b, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:10:29,108 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf2/58e66d0dabaf481cb438e9ead062860d as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/58e66d0dabaf481cb438e9ead062860d 2023-07-12 19:10:29,113 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/58e66d0dabaf481cb438e9ead062860d, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:10:29,114 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for d0505bae67c35c958e3be134081a0fc8 in 44ms, sequenceid=6, compaction requested=false 2023-07-12 19:10:29,114 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-07-12 19:10:29,114 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0505bae67c35c958e3be134081a0fc8: 2023-07-12 19:10:29,114 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:29,114 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:29,115 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:29,115 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,115 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:29,115 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:29,116 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,116 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:29,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:29,116 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,116 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:29,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:29,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:29,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:29,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-12 19:10:29,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31284872[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:29,118 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-12 19:10:29,119 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,119 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,119 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,119 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,120 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,120 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:29,120 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:29,120 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:29,120 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:29,120 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:29,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,120 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:29,120 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:29,120 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:29,120 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:29,120 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:29,121 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:29,121 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,121 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:29,122 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:29,122 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,122 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:29,123 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,123 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:29,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:29,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:29,123 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,123 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:29,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:29,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:29,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:29,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:29,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-12 19:10:29,127 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-12 19:10:29,127 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:29,127 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:29,127 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-12 19:10:29,127 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:29,128 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,128 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:29,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:29,128 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,128 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,128 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,128 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:29,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,129 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,128 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:29,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,129 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,129 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:29,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,131 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,131 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,131 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,135 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,135 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,135 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,135 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,135 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,136 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,136 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,135 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,135 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,136 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-12 19:10:29,137 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:29,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,136 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,137 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:29,137 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,137 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:29,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,137 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,138 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,138 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,138 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,138 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,138 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,138 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,238 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:29,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:29,268 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:29,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:29,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-12 19:10:29,270 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-12 19:10:29,270 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,271 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-12 19:10:29,271 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-12 19:10:29,271 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,271 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,272 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,272 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,272 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,272 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,273 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,273 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:29,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:29,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:29,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:29,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:29,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,274 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 19:10:29,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:29,274 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:29,274 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 19:10:29,275 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 19:10:29,275 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:29,275 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:29,275 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:29,275 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,275 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,276 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:29,275 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:29,277 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:29,277 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:29,277 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:29,277 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:29,277 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,278 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:29,278 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:29,278 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:29,278 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:29,278 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. started... 2023-07-12 19:10:29,278 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:29,278 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:29,278 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0505bae67c35c958e3be134081a0fc8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:29,285 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,286 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,286 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,287 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:29,287 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,287 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:29,292 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf1/246096b68ff2402b9dcb312a377d81c3 2023-07-12 19:10:29,309 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf2/0c1a752213484b179c6458186fd9482b 2023-07-12 19:10:29,314 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf1/246096b68ff2402b9dcb312a377d81c3 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/246096b68ff2402b9dcb312a377d81c3 2023-07-12 19:10:29,319 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/246096b68ff2402b9dcb312a377d81c3, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:10:29,320 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf2/0c1a752213484b179c6458186fd9482b as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/0c1a752213484b179c6458186fd9482b 2023-07-12 19:10:29,326 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/0c1a752213484b179c6458186fd9482b, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:10:29,328 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d0505bae67c35c958e3be134081a0fc8 in 50ms, sequenceid=19, compaction requested=false 2023-07-12 19:10:29,328 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0505bae67c35c958e3be134081a0fc8: 2023-07-12 19:10:29,328 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:29,328 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:29,328 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:29,328 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,328 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:29,328 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:29,329 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,329 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:29,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:29,330 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,330 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:29,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:29,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:29,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-12 19:10:29,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25e619c9[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:29,332 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-12 19:10:29,333 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,333 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,333 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,333 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,334 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:29,334 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,334 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:29,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,334 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:29,334 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:29,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,334 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:29,334 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:29,334 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:29,334 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:29,334 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:29,334 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:29,336 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:29,336 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,336 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:29,337 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:29,337 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,337 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:29,338 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:29,338 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,338 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:29,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:29,338 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:29,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:29,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:29,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:29,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:29,342 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-12 19:10:29,342 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-12 19:10:29,342 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:29,342 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:29,342 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-12 19:10:29,342 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:29,343 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,343 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,343 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,343 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,343 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:29,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:29,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,344 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,344 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,344 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:29,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,345 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,345 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:29,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:29,347 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,348 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,348 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,351 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,351 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,351 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,351 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,352 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,352 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,351 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,353 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,352 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-12 19:10:29,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:29,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,351 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,353 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,353 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,353 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,353 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,353 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,353 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,353 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,353 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,353 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,353 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:29,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,354 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:29,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,454 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:29,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:29,481 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:29,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:29,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-12 19:10:29,483 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-12 19:10:29,483 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,484 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-12 19:10:29,484 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-12 19:10:29,484 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,485 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,485 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,486 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,486 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,486 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,486 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,486 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:29,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,486 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,487 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,487 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:29,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,487 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:29,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:29,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:29,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,487 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:29,487 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 19:10:29,487 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:29,487 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:29,487 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 19:10:29,487 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:29,488 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,487 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:29,488 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:29,488 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,490 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 19:10:29,491 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:29,491 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:29,491 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:29,491 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:29,491 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:29,491 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:29,492 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:29,491 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:29,492 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,491 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:29,492 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:29,492 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:29,492 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. started... 2023-07-12 19:10:29,492 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:29,493 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0505bae67c35c958e3be134081a0fc8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:29,495 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,495 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,495 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,495 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:29,496 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,496 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:29,507 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf1/7f49c040019b4ea8a74a717a4da2b9d9 2023-07-12 19:10:29,526 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf2/31b19a3025614e018c92f7d6b1cd6c9a 2023-07-12 19:10:29,532 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf1/7f49c040019b4ea8a74a717a4da2b9d9 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/7f49c040019b4ea8a74a717a4da2b9d9 2023-07-12 19:10:29,538 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/7f49c040019b4ea8a74a717a4da2b9d9, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:10:29,539 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf2/31b19a3025614e018c92f7d6b1cd6c9a as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/31b19a3025614e018c92f7d6b1cd6c9a 2023-07-12 19:10:29,545 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/31b19a3025614e018c92f7d6b1cd6c9a, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:10:29,546 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d0505bae67c35c958e3be134081a0fc8 in 53ms, sequenceid=32, compaction requested=true 2023-07-12 19:10:29,546 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0505bae67c35c958e3be134081a0fc8: 2023-07-12 19:10:29,546 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:29,547 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:29,547 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:29,547 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,547 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:29,547 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:29,548 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,548 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:29,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:29,548 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,548 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:29,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:29,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:29,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:29,550 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-12 19:10:29,550 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@304e60cb[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:29,550 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-12 19:10:29,551 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,551 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,551 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,551 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,552 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,552 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:29,552 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:29,552 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:29,552 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:29,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,552 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:29,552 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:29,552 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:29,552 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:29,552 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:29,552 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:29,553 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:29,553 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,553 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:29,554 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:29,554 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:29,554 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,554 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:29,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:29,554 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:29,554 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,555 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:29,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:29,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:29,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:29,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:29,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-12 19:10:29,559 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-12 19:10:29,559 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:29,559 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:29,559 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-12 19:10:29,559 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:29,569 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,569 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:29,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:29,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,569 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,569 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,569 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,569 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,569 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:29,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,569 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,570 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,570 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:29,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,572 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:29,572 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,572 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,577 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,577 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,577 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,577 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,577 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,577 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-12 19:10:29,578 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:29,577 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,577 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,578 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,578 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,577 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,577 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,578 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,579 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,578 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:29,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,579 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,579 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,579 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:29,579 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,580 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,680 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:29,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:29,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:29,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:29,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-12 19:10:29,705 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-12 19:10:29,705 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,706 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-12 19:10:29,706 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-12 19:10:29,706 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,706 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,708 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,708 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,708 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,708 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,709 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,709 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:29,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,709 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,709 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,709 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:29,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,710 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:29,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:29,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:29,710 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:29,710 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 19:10:29,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:29,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:29,712 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:29,712 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 19:10:29,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:29,712 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:29,713 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 19:10:29,713 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:29,713 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:29,713 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:29,713 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:29,713 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:29,713 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:29,714 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,713 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:29,713 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:29,713 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:29,715 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. started... 2023-07-12 19:10:29,715 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:29,715 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:29,715 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:29,715 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0505bae67c35c958e3be134081a0fc8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:29,720 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,720 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,720 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,720 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:29,720 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,720 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:29,728 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf1/bd0eba394a11405c910688e18771a4cc 2023-07-12 19:10:29,743 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf2/e61e803e20d24b718b6aa65468c1f60c 2023-07-12 19:10:29,749 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf1/bd0eba394a11405c910688e18771a4cc as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bd0eba394a11405c910688e18771a4cc 2023-07-12 19:10:29,756 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bd0eba394a11405c910688e18771a4cc, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:10:29,758 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf2/e61e803e20d24b718b6aa65468c1f60c as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/e61e803e20d24b718b6aa65468c1f60c 2023-07-12 19:10:29,765 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/e61e803e20d24b718b6aa65468c1f60c, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:10:29,766 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d0505bae67c35c958e3be134081a0fc8 in 51ms, sequenceid=45, compaction requested=true 2023-07-12 19:10:29,766 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0505bae67c35c958e3be134081a0fc8: 2023-07-12 19:10:29,766 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:29,766 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:29,766 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:29,766 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,766 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:29,766 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:29,767 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,767 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:29,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:29,768 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,768 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:29,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:29,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:29,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:29,770 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-12 19:10:29,770 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47ed7315[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:29,770 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-12 19:10:29,770 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,771 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,771 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,771 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,771 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,771 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:29,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,771 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:29,771 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:29,771 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:29,771 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:29,771 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:29,771 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:29,771 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:29,771 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:29,771 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:29,772 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,772 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:29,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:29,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:29,772 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,772 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:29,773 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:29,773 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:29,773 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:29,773 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,773 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:29,774 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:29,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:29,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,776 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-12 19:10:29,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,776 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-12 19:10:29,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:29,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,777 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:29,777 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-07-12 19:10:29,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:29,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-12 19:10:29,778 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-12 19:10:29,778 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:29,778 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:29,778 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-12 19:10:29,778 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:29,785 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,785 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:29,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:29,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,785 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,785 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,785 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,786 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,786 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,786 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:29,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,786 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,787 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:29,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,789 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:29,789 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,789 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,803 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,803 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,803 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,803 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,803 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,803 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,804 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-12 19:10:29,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:29,803 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,803 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,803 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,804 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:29,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,804 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,805 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,805 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:29,805 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,805 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,806 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,806 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,806 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,806 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,905 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:29,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:29,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:29,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:29,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-12 19:10:29,935 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-12 19:10:29,935 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,935 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-12 19:10:29,935 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-12 19:10:29,936 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,936 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,937 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,937 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,937 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:29,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,937 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:29,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:29,937 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,937 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:29,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:29,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:29,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:29,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:29,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:29,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:29,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:29,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 19:10:29,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:29,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 19:10:29,938 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:29,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:29,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 19:10:29,939 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:29,939 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,938 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,939 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:29,939 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:29,939 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:29,939 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:29,939 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:29,939 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:29,939 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:29,939 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:29,939 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:29,939 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:29,939 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:29,939 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:29,939 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:29,939 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. started... 2023-07-12 19:10:29,939 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0505bae67c35c958e3be134081a0fc8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:29,940 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,940 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,940 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,940 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:29,940 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,940 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:29,952 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf1/5701a846a9024ee99cb023d93ddf021f 2023-07-12 19:10:29,968 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf2/ceb560de5aec4f6dac7ad5e9c558d976 2023-07-12 19:10:29,974 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf1/5701a846a9024ee99cb023d93ddf021f as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5701a846a9024ee99cb023d93ddf021f 2023-07-12 19:10:29,978 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5701a846a9024ee99cb023d93ddf021f, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:10:29,979 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf2/ceb560de5aec4f6dac7ad5e9c558d976 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/ceb560de5aec4f6dac7ad5e9c558d976 2023-07-12 19:10:29,985 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/ceb560de5aec4f6dac7ad5e9c558d976, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:10:29,986 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d0505bae67c35c958e3be134081a0fc8 in 47ms, sequenceid=58, compaction requested=true 2023-07-12 19:10:29,986 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0505bae67c35c958e3be134081a0fc8: 2023-07-12 19:10:29,986 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:29,986 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:29,986 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:29,986 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,986 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:29,986 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:29,988 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,988 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:29,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:29,988 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,988 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:29,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:29,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:29,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:29,991 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-12 19:10:29,991 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34f5c842[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:29,991 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-12 19:10:29,991 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,991 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,991 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,991 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,992 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:29,992 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:29,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,992 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:29,992 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,992 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:29,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:29,992 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:29,992 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:29,992 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:29,992 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:29,992 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:29,992 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:29,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,993 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:29,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:29,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:29,993 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,993 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:29,993 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:29,993 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,993 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:29,993 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:29,993 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:29,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:29,993 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:29,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:29,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:29,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:29,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:29,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:29,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-12 19:10:29,997 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-12 19:10:29,997 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:29,997 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:29,997 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-12 19:10:29,997 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:29,999 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,999 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:29,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:29,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,999 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,999 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,999 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,999 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,999 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:29,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:29,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,999 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:29,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:29,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:29,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:29,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:30,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:29,999 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:30,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:30,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:30,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:30,000 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:30,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:30,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:30,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:30,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:30,001 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:30,002 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:30,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,002 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:30,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:30,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:30,011 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:30,011 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:30,011 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:30,011 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:30,011 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:30,011 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:30,011 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:30,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:30,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:30,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:30,012 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:30,012 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:30,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-12 19:10:30,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:30,011 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:30,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:30,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:30,012 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:30,012 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:30,012 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:30,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:30,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:30,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:30,012 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:30,012 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:30,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:30,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:30,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:30,013 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,013 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:30,013 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,013 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,013 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,113 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:30,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:30,139 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:30,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:30,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-12 19:10:30,142 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-12 19:10:30,142 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:30,142 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-12 19:10:30,142 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-12 19:10:30,142 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,142 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:30,143 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:30,143 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:30,143 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:30,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:30,143 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:30,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:30,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:30,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:30,144 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:30,144 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:30,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:30,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:30,144 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:30,144 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:30,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:30,144 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:30,145 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:30,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:30,145 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:30,145 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 19:10:30,145 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 19:10:30,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:30,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:30,145 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:30,145 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 19:10:30,145 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:30,145 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:30,145 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:30,145 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 19:10:30,145 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:30,145 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 19:10:30,145 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:30,145 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 19:10:30,145 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:30,145 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:30,146 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:30,146 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:30,146 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:30,146 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:30,146 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:30,146 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 19:10:30,146 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:30,146 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:30,146 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. started... 2023-07-12 19:10:30,146 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:30,146 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0505bae67c35c958e3be134081a0fc8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:30,147 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,147 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,147 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,147 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:30,147 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,148 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:30,158 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf1/bc3adf313ea64f7d90c1145883882006 2023-07-12 19:10:30,173 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf2/bb802536c2224693bd5ede242b652340 2023-07-12 19:10:30,178 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf1/bc3adf313ea64f7d90c1145883882006 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bc3adf313ea64f7d90c1145883882006 2023-07-12 19:10:30,183 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bc3adf313ea64f7d90c1145883882006, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:10:30,184 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf2/bb802536c2224693bd5ede242b652340 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/bb802536c2224693bd5ede242b652340 2023-07-12 19:10:30,191 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/bb802536c2224693bd5ede242b652340, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:10:30,192 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d0505bae67c35c958e3be134081a0fc8 in 46ms, sequenceid=71, compaction requested=true 2023-07-12 19:10:30,192 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0505bae67c35c958e3be134081a0fc8: 2023-07-12 19:10:30,192 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:30,192 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:30,192 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:30,192 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,192 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 19:10:30,192 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 19:10:30,194 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,194 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:30,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:30,194 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,194 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:30,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:30,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:30,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:30,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:30,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:30,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:30,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-12 19:10:30,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21c8b744[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:30,197 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-12 19:10:30,197 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,198 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,198 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,198 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,198 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:30,199 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,199 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:30,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,199 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:30,199 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:30,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,199 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:30,199 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 19:10:30,199 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:30,199 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 19:10:30,199 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:30,199 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:30,200 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:30,200 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:30,200 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:30,201 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,201 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:30,201 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:30,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:30,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:30,201 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:30,201 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 19:10:30,201 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:30,201 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 19:10:30,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:30,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:30,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:30,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:30,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:30,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:30,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:30,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:30,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:30,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:30,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-12 19:10:30,205 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-12 19:10:30,205 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:30,205 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:30,205 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-12 19:10:30,206 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:30,207 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,207 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,207 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:30,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:30,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,207 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:30,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,207 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:30,207 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:30,208 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:30,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:30,208 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:30,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:30,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:30,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:30,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:30,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:30,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,209 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:30,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:30,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,210 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:30,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:30,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:30,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:30,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:30,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 19:10:30,211 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,211 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:30,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:30,211 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:30,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:30,213 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:30,213 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:30,213 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:30,213 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:30,213 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,213 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:30,213 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:30,213 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:30,213 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:30,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:30,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:30,213 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:30,213 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 19:10:30,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:30,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:30,213 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:30,213 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-12 19:10:30,214 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:30,213 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:30,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:30,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:30,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:30,214 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:30,214 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:30,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:30,214 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,214 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:30,214 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:30,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:30,214 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 19:10:30,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:30,214 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 19:10:30,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:30,314 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:30,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:30,587 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-07-12 19:10:30,590 DEBUG [Listener at localhost.localdomain/40405] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:30,593 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:30,595 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:30,595 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:30,596 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:30,597 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:30,597 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:30,597 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:30,598 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:30,844 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:30,845 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:30,845 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:30,845 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:30,847 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-07-12 19:10:30,848 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:30,848 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:30,848 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:30,849 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:30,849 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 19:10:30,853 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-07-12 19:10:30,854 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:30,855 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:30,855 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:30,855 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:30,856 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:30,856 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:30,857 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:30,857 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:30,858 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:30,858 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:30,859 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:30,859 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:30,859 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:30,859 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:30,859 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-07-12 19:10:30,860 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:30,861 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:30,861 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189060861 2023-07-12 19:10:30,863 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:30,863 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:30,867 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:30,867 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=d0505bae67c35c958e3be134081a0fc8, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,38353,1689189020667, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667, openSeqNum=2 2023-07-12 19:10:30,868 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:30,868 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189090868 2023-07-12 19:10:30,868 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189150868 2023-07-12 19:10:30,868 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 19:10:30,868 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:30,870 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:30,871 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:30,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:30,871 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:31,118 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,118 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:31,118 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:31,119 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:31,120 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:31,120 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:31,120 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:31,120 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:31,120 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:31,121 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 19:10:31,124 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-12 19:10:31,125 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:31,125 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:31,126 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:31,126 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:31,127 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:31,128 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:31,128 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:31,128 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:31,129 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:31,130 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,130 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:31,130 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,130 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,130 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,130 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-12 19:10:31,131 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:31,131 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:31,132 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189061132 2023-07-12 19:10:31,133 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:31,133 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,135 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:31,136 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=d0505bae67c35c958e3be134081a0fc8, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,38353,1689189020667, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667, openSeqNum=2 2023-07-12 19:10:31,136 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:31,136 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189091136 2023-07-12 19:10:31,136 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189151136 2023-07-12 19:10:31,136 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 19:10:31,136 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:31,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN}] 2023-07-12 19:10:31,139 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:31,139 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:31,139 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:31,386 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,387 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:31,387 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:31,387 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:31,387 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:31,388 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:31,388 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:31,388 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:31,388 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:31,389 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:31,392 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-12 19:10:31,393 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:31,394 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:31,394 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:31,394 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:31,396 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:31,396 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:31,397 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:31,397 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:31,398 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:31,398 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,398 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:31,398 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,398 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,399 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,399 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-07-12 19:10:31,399 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:31,400 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:31,400 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189061400 2023-07-12 19:10:31,402 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,402 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:31,404 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:31,405 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=d0505bae67c35c958e3be134081a0fc8, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,38353,1689189020667, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667, openSeqNum=2 2023-07-12 19:10:31,405 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:31,405 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189091405 2023-07-12 19:10:31,405 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189151405 2023-07-12 19:10:31,405 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:10:31,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN}] 2023-07-12 19:10:31,406 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:31,408 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN 2023-07-12 19:10:31,409 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=d0505bae67c35c958e3be134081a0fc8, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:31,409 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689189031409"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189031409"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189031409"}]},"ts":"1689189031409"} 2023-07-12 19:10:31,410 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:31,410 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:31,411 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN 2023-07-12 19:10:31,411 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:31,411 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:31,655 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,656 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:31,656 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:31,656 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:31,656 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:31,657 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:31,657 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:31,657 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:31,657 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:31,659 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:31,663 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 19:10:31,664 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-12 19:10:31,665 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:31,666 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:31,667 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:31,667 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:31,667 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:31,668 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:31,668 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:31,668 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:31,668 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:31,669 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,669 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN 2023-07-12 19:10:31,669 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:31,669 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,670 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,670 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,670 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:31,670 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN 2023-07-12 19:10:31,670 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-07-12 19:10:31,671 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:31,671 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:31,671 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8 to restore RIT 2023-07-12 19:10:31,672 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189061672 2023-07-12 19:10:31,672 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN 2023-07-12 19:10:31,673 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:31,675 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:31,675 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=d0505bae67c35c958e3be134081a0fc8, regionState=CLOSING, lastHost=jenkins-hbase20.apache.org,38353,1689189020667, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667, openSeqNum=2 2023-07-12 19:10:31,675 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:31,676 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189091676 2023-07-12 19:10:31,676 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189151676 2023-07-12 19:10:31,676 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:10:31,676 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=d0505bae67c35c958e3be134081a0fc8, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:31,676 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689189031676"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189031676"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189031676"}]},"ts":"1689189031676"} 2023-07-12 19:10:31,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:31,678 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:31,832 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:31,832 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing d0505bae67c35c958e3be134081a0fc8, disabling compactions & flushes 2023-07-12 19:10:31,832 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:31,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:31,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. after waiting 0 ms 2023-07-12 19:10:31,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:31,833 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing d0505bae67c35c958e3be134081a0fc8 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-12 19:10:31,850 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1 2023-07-12 19:10:31,869 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf2/f2f6fbf463104434a65a704f32849165 2023-07-12 19:10:31,875 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1 2023-07-12 19:10:31,880 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 19:10:31,881 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/.tmp/cf2/f2f6fbf463104434a65a704f32849165 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/f2f6fbf463104434a65a704f32849165 2023-07-12 19:10:31,888 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/f2f6fbf463104434a65a704f32849165, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 19:10:31,890 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d0505bae67c35c958e3be134081a0fc8 in 56ms, sequenceid=82, compaction requested=true 2023-07-12 19:10:31,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-12 19:10:31,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:31,915 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. 2023-07-12 19:10:31,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for d0505bae67c35c958e3be134081a0fc8: 2023-07-12 19:10:31,918 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:31,918 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=d0505bae67c35c958e3be134081a0fc8, regionState=CLOSED 2023-07-12 19:10:31,918 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689189031918"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189031918"}]},"ts":"1689189031918"} 2023-07-12 19:10:31,920 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:31,920 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:31,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:31,920 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:31,926 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,926 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:31,926 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:31,926 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:31,927 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:31,927 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:31,927 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:31,927 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:31,927 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:31,928 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 19:10:31,929 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 19:10:31,929 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-12 19:10:31,929 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-07-12 19:10:31,930 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:31,930 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:31,930 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:31,930 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:31,931 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:31,931 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:31,931 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:31,932 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:31,932 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:31,933 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,933 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN 2023-07-12 19:10:31,933 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:31,933 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:31,933 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,933 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,933 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:31,933 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:31,933 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN 2023-07-12 19:10:31,933 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:31,933 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-12 19:10:31,934 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:31,935 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:31,935 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8 to restore RIT 2023-07-12 19:10:31,935 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189061935 2023-07-12 19:10:31,936 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:31,936 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:31,938 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:31,939 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=d0505bae67c35c958e3be134081a0fc8, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,38353,1689189020667, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667, openSeqNum=2 2023-07-12 19:10:31,939 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:31,939 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189091939 2023-07-12 19:10:31,939 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189151939 2023-07-12 19:10:31,939 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:10:31,939 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=d0505bae67c35c958e3be134081a0fc8, regionState=CLOSED, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:31,939 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689189031939"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189031939"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189031939"}]},"ts":"1689189031939"} 2023-07-12 19:10:31,941 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:31,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-07-12 19:10:31,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 in 263 msec 2023-07-12 19:10:31,944 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:31,944 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN 2023-07-12 19:10:31,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:31,944 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:32,189 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,189 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:32,189 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:32,190 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:32,190 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:32,190 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:32,190 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:32,190 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:32,190 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:32,191 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 19:10:32,192 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 19:10:32,192 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-12 19:10:32,192 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:10:32,193 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:32,193 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:32,193 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:32,194 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:32,194 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:32,194 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:32,195 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:32,195 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:32,195 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:32,196 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,196 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN 2023-07-12 19:10:32,196 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:32,196 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:32,197 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,197 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,197 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,197 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:32,197 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:32,197 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 19:10:32,198 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:32,198 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:32,199 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8 to restore RIT 2023-07-12 19:10:32,199 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189062199 2023-07-12 19:10:32,200 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:32,200 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN 2023-07-12 19:10:32,202 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:32,202 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=d0505bae67c35c958e3be134081a0fc8, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,38353,1689189020667, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667, openSeqNum=2 2023-07-12 19:10:32,202 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:32,202 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189092202 2023-07-12 19:10:32,202 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189152202 2023-07-12 19:10:32,203 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 19:10:32,203 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN 2023-07-12 19:10:32,204 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:32,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-07-12 19:10:32,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN in 799 msec 2023-07-12 19:10:32,210 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:32,210 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:32,210 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:32,453 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,453 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:32,453 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:32,453 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:32,454 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:32,454 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:32,454 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:32,454 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:32,455 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:32,456 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:32,458 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 19:10:32,458 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:10:32,459 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:10:32,459 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:32,460 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:32,460 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:32,460 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:32,461 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:32,462 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:32,462 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:32,462 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:32,463 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:32,464 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,464 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN 2023-07-12 19:10:32,464 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:32,465 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:32,465 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:32,465 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:32,465 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-07-12 19:10:32,466 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:32,466 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:32,467 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189062467 2023-07-12 19:10:32,467 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:32,468 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,470 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:32,471 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=d0505bae67c35c958e3be134081a0fc8, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,38353,1689189020667, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667, openSeqNum=2 2023-07-12 19:10:32,471 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:32,471 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189092471 2023-07-12 19:10:32,471 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189152471 2023-07-12 19:10:32,471 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:10:32,471 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,471 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,471 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,476 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:32,488 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=d0505bae67c35c958e3be134081a0fc8, threads=14 2023-07-12 19:10:32,489 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/0c1a752213484b179c6458186fd9482b for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,489 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/31b19a3025614e018c92f7d6b1cd6c9a for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,489 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/58e66d0dabaf481cb438e9ead062860d for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,489 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/bb802536c2224693bd5ede242b652340 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,490 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/ceb560de5aec4f6dac7ad5e9c558d976 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,490 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/e61e803e20d24b718b6aa65468c1f60c for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,490 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/f2f6fbf463104434a65a704f32849165 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,490 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,494 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/246096b68ff2402b9dcb312a377d81c3 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,494 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5701a846a9024ee99cb023d93ddf021f for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,495 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5aef80d484ce4cd6b6fc892fa133a77b for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,495 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/7f49c040019b4ea8a74a717a4da2b9d9 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,498 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bc3adf313ea64f7d90c1145883882006 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,498 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bd0eba394a11405c910688e18771a4cc for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,608 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/bb802536c2224693bd5ede242b652340 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,619 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/e61e803e20d24b718b6aa65468c1f60c for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,631 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5701a846a9024ee99cb023d93ddf021f for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,632 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/0c1a752213484b179c6458186fd9482b for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,633 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/7f49c040019b4ea8a74a717a4da2b9d9 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,634 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,635 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bc3adf313ea64f7d90c1145883882006 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,645 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bd0eba394a11405c910688e18771a4cc for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,645 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/31b19a3025614e018c92f7d6b1cd6c9a for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,655 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/f2f6fbf463104434a65a704f32849165 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,658 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/ceb560de5aec4f6dac7ad5e9c558d976 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,660 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/246096b68ff2402b9dcb312a377d81c3 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,661 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5aef80d484ce4cd6b6fc892fa133a77b for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,661 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/58e66d0dabaf481cb438e9ead062860d for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,661 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region d0505bae67c35c958e3be134081a0fc8 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 19:10:32,703 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:32,703 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,704 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:32,704 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:32,721 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,721 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:32,722 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:32,722 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:32,723 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:32,724 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:32,724 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:32,724 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:32,725 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:32,726 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:32,730 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-12 19:10:32,730 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:10:32,730 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:10:32,731 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:32,731 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:32,731 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:32,732 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:32,732 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:32,733 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:32,733 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:32,734 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:32,734 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:32,735 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,735 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN 2023-07-12 19:10:32,735 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:32,735 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:32,736 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,736 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,736 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,736 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:32,736 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:32,736 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-07-12 19:10:32,737 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:32,737 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:32,738 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189062738 2023-07-12 19:10:32,739 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:32,739 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:32,742 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:32,742 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=d0505bae67c35c958e3be134081a0fc8, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,38353,1689189020667, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667, openSeqNum=2 2023-07-12 19:10:32,742 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:32,742 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189092742 2023-07-12 19:10:32,742 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189152742 2023-07-12 19:10:32,743 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:10:32,753 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=d0505bae67c35c958e3be134081a0fc8, threads=14 2023-07-12 19:10:32,754 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/0c1a752213484b179c6458186fd9482b for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,754 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/31b19a3025614e018c92f7d6b1cd6c9a for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,754 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/58e66d0dabaf481cb438e9ead062860d for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,755 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/bb802536c2224693bd5ede242b652340 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,755 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/ceb560de5aec4f6dac7ad5e9c558d976 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,755 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/e61e803e20d24b718b6aa65468c1f60c for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,755 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/f2f6fbf463104434a65a704f32849165 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,755 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,755 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/246096b68ff2402b9dcb312a377d81c3 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,756 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5701a846a9024ee99cb023d93ddf021f for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,756 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5aef80d484ce4cd6b6fc892fa133a77b for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,756 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/7f49c040019b4ea8a74a717a4da2b9d9 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,756 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bc3adf313ea64f7d90c1145883882006 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,758 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bd0eba394a11405c910688e18771a4cc for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,890 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/ceb560de5aec4f6dac7ad5e9c558d976 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,923 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bc3adf313ea64f7d90c1145883882006 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,938 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/e61e803e20d24b718b6aa65468c1f60c for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,943 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/246096b68ff2402b9dcb312a377d81c3 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,949 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/0c1a752213484b179c6458186fd9482b for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,951 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bd0eba394a11405c910688e18771a4cc for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,951 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,953 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/7f49c040019b4ea8a74a717a4da2b9d9 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,953 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/bb802536c2224693bd5ede242b652340 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,954 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/31b19a3025614e018c92f7d6b1cd6c9a for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,962 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/f2f6fbf463104434a65a704f32849165 for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,963 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/58e66d0dabaf481cb438e9ead062860d for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:32,967 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5aef80d484ce4cd6b6fc892fa133a77b for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:33,290 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 19:10:33,368 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5701a846a9024ee99cb023d93ddf021f for region: d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:33,368 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region d0505bae67c35c958e3be134081a0fc8 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 19:10:33,406 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:33,413 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 19:10:33,415 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 19:10:33,415 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:33,415 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:33,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:33,416 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:33,493 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:33,493 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:33,493 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:33,493 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:33,494 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:33,494 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:33,494 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:33,494 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:33,494 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:33,495 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 19:10:33,496 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-12 19:10:33,496 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:10:33,496 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:10:33,496 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:33,497 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:33,497 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:33,497 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:33,498 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:33,498 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:33,498 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:33,498 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:33,499 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:33,500 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:33,500 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN 2023-07-12 19:10:33,500 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:33,500 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:33,501 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:33,501 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:33,501 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:33,501 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:33,501 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:33,501 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 19:10:33,502 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:33,503 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:33,503 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189063503 2023-07-12 19:10:33,504 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:33,504 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:33,507 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:33,508 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=d0505bae67c35c958e3be134081a0fc8, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,38353,1689189020667, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667, openSeqNum=2 2023-07-12 19:10:33,508 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:33,508 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189093508 2023-07-12 19:10:33,508 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189153508 2023-07-12 19:10:33,508 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 19:10:33,511 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:33,514 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:33,514 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:33,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:33,514 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:33,758 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:10:33,758 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:33,758 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:33,759 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:33,759 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:33,759 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:33,759 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:33,759 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:33,759 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:33,760 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 19:10:33,761 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-12 19:10:33,761 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:10:33,761 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:10:33,762 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:33,762 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:33,762 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:33,763 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:33,763 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:33,764 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:33,764 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:33,764 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:33,765 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:33,765 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:33,766 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN 2023-07-12 19:10:33,766 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:33,766 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:33,766 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:33,766 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:33,766 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:33,766 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d0505bae67c35c958e3be134081a0fc8, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:33,767 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d0505bae67c35c958e3be134081a0fc8, server=jenkins-hbase20.apache.org,38353,1689189020667 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:33,767 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 19:10:33,768 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:33,769 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:33,769 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189063769 2023-07-12 19:10:33,770 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:33,771 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:33,774 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:33,774 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=d0505bae67c35c958e3be134081a0fc8, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,38353,1689189020667, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667, openSeqNum=2 2023-07-12 19:10:33,774 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:33,774 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189093774 2023-07-12 19:10:33,774 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189153774 2023-07-12 19:10:33,775 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-07-12 19:10:33,778 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689189033777"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1689189033777"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1689189033777"}]},"ts":"1689189033777"} 2023-07-12 19:10:33,778 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189033777"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189033777"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189033777"}]},"ts":"1689189033777"} 2023-07-12 19:10:33,778 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189033777"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189033777"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189033777"}]},"ts":"1689189033777"} 2023-07-12 19:10:33,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0e9dac5a8ee81c6ad08618940a5f514, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18a1a0a446e655ef02aa7598aabe8703, ASSIGN}] 2023-07-12 19:10:33,787 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18a1a0a446e655ef02aa7598aabe8703, ASSIGN 2023-07-12 19:10:33,788 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18a1a0a446e655ef02aa7598aabe8703, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,38353,1689189020667; forceNewPlan=false, retain=false 2023-07-12 19:10:33,789 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0e9dac5a8ee81c6ad08618940a5f514, ASSIGN 2023-07-12 19:10:33,791 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0e9dac5a8ee81c6ad08618940a5f514, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,38353,1689189020667; forceNewPlan=false, retain=false 2023-07-12 19:10:33,939 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-12 19:10:33,942 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=18a1a0a446e655ef02aa7598aabe8703, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:33,943 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189033942"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189033942"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189033942"}]},"ts":"1689189033942"} 2023-07-12 19:10:33,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure 18a1a0a446e655ef02aa7598aabe8703, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:33,951 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=c0e9dac5a8ee81c6ad08618940a5f514, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:33,952 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189033951"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189033951"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189033951"}]},"ts":"1689189033951"} 2023-07-12 19:10:33,953 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure c0e9dac5a8ee81c6ad08618940a5f514, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:34,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514. 2023-07-12 19:10:34,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c0e9dac5a8ee81c6ad08618940a5f514, NAME => 'testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.', STARTKEY => '', ENDKEY => '41'} 2023-07-12 19:10:34,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:34,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR c0e9dac5a8ee81c6ad08618940a5f514 2023-07-12 19:10:34,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:34,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for c0e9dac5a8ee81c6ad08618940a5f514 2023-07-12 19:10:34,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for c0e9dac5a8ee81c6ad08618940a5f514 2023-07-12 19:10:34,106 INFO [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c0e9dac5a8ee81c6ad08618940a5f514 2023-07-12 19:10:34,107 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1 2023-07-12 19:10:34,107 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1 2023-07-12 19:10:34,108 INFO [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0e9dac5a8ee81c6ad08618940a5f514 columnFamilyName cf1 2023-07-12 19:10:34,125 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1-bottom 2023-07-12 19:10:34,130 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/246096b68ff2402b9dcb312a377d81c3-bottom 2023-07-12 19:10:34,134 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5701a846a9024ee99cb023d93ddf021f-bottom 2023-07-12 19:10:34,140 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5aef80d484ce4cd6b6fc892fa133a77b-bottom 2023-07-12 19:10:34,145 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/7f49c040019b4ea8a74a717a4da2b9d9-bottom 2023-07-12 19:10:34,150 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bc3adf313ea64f7d90c1145883882006-bottom 2023-07-12 19:10:34,154 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bd0eba394a11405c910688e18771a4cc-bottom 2023-07-12 19:10:34,155 INFO [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(310): Store=c0e9dac5a8ee81c6ad08618940a5f514/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:34,155 INFO [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c0e9dac5a8ee81c6ad08618940a5f514 2023-07-12 19:10:34,155 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2 2023-07-12 19:10:34,156 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2 2023-07-12 19:10:34,156 INFO [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0e9dac5a8ee81c6ad08618940a5f514 columnFamilyName cf2 2023-07-12 19:10:34,174 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/0c1a752213484b179c6458186fd9482b-bottom 2023-07-12 19:10:34,181 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/31b19a3025614e018c92f7d6b1cd6c9a-bottom 2023-07-12 19:10:34,186 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/58e66d0dabaf481cb438e9ead062860d-bottom 2023-07-12 19:10:34,192 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/bb802536c2224693bd5ede242b652340-bottom 2023-07-12 19:10:34,198 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/ceb560de5aec4f6dac7ad5e9c558d976-bottom 2023-07-12 19:10:34,205 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/e61e803e20d24b718b6aa65468c1f60c-bottom 2023-07-12 19:10:34,213 DEBUG [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/f2f6fbf463104434a65a704f32849165-bottom 2023-07-12 19:10:34,214 INFO [StoreOpener-c0e9dac5a8ee81c6ad08618940a5f514-1] regionserver.HStore(310): Store=c0e9dac5a8ee81c6ad08618940a5f514/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:34,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514 2023-07-12 19:10:34,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514 2023-07-12 19:10:34,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:10:34,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for c0e9dac5a8ee81c6ad08618940a5f514 2023-07-12 19:10:34,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened c0e9dac5a8ee81c6ad08618940a5f514; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63452293, jitterRate=-0.05448715388774872}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:10:34,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for c0e9dac5a8ee81c6ad08618940a5f514: 2023-07-12 19:10:34,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514., pid=34, masterSystemTime=1689189034100 2023-07-12 19:10:34,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:10:34,223 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:10:34,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:10:34,223 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:10:34,223 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514. 2023-07-12 19:10:34,223 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1912): c0e9dac5a8ee81c6ad08618940a5f514/cf1 is initiating minor compaction (all files) 2023-07-12 19:10:34,223 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514. 2023-07-12 19:10:34,223 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1912): c0e9dac5a8ee81c6ad08618940a5f514/cf2 is initiating minor compaction (all files) 2023-07-12 19:10:34,223 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c0e9dac5a8ee81c6ad08618940a5f514/cf1 in testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514. 2023-07-12 19:10:34,223 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2259): Starting compaction of c0e9dac5a8ee81c6ad08618940a5f514/cf2 in testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514. 2023-07-12 19:10:34,223 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/58e66d0dabaf481cb438e9ead062860d-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/0c1a752213484b179c6458186fd9482b-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/31b19a3025614e018c92f7d6b1cd6c9a-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/e61e803e20d24b718b6aa65468c1f60c-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/ceb560de5aec4f6dac7ad5e9c558d976-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/bb802536c2224693bd5ede242b652340-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/f2f6fbf463104434a65a704f32849165-bottom] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/.tmp, totalSize=34.9 K 2023-07-12 19:10:34,223 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5aef80d484ce4cd6b6fc892fa133a77b-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/246096b68ff2402b9dcb312a377d81c3-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/7f49c040019b4ea8a74a717a4da2b9d9-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bd0eba394a11405c910688e18771a4cc-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5701a846a9024ee99cb023d93ddf021f-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bc3adf313ea64f7d90c1145883882006-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1-bottom] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/.tmp, totalSize=34.9 K 2023-07-12 19:10:34,224 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689189029044 2023-07-12 19:10:34,224 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689189029044 2023-07-12 19:10:34,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514. 2023-07-12 19:10:34,225 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689189029243 2023-07-12 19:10:34,225 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689189029243 2023-07-12 19:10:34,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514. 2023-07-12 19:10:34,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703. 2023-07-12 19:10:34,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 18a1a0a446e655ef02aa7598aabe8703, NAME => 'testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.', STARTKEY => '41', ENDKEY => ''} 2023-07-12 19:10:34,225 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689189029458 2023-07-12 19:10:34,225 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689189029458 2023-07-12 19:10:34,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:34,225 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=c0e9dac5a8ee81c6ad08618940a5f514, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:34,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:34,226 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189034225"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189034225"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189034225"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189034225"}]},"ts":"1689189034225"} 2023-07-12 19:10:34,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:34,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:34,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:34,226 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689189029682 2023-07-12 19:10:34,226 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689189029682 2023-07-12 19:10:34,226 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689189029910 2023-07-12 19:10:34,227 INFO [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:34,227 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689189030116 2023-07-12 19:10:34,227 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689189029910 2023-07-12 19:10:34,227 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689189030116 2023-07-12 19:10:34,227 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689189030320 2023-07-12 19:10:34,228 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1 2023-07-12 19:10:34,228 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1 2023-07-12 19:10:34,228 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689189030320 2023-07-12 19:10:34,228 INFO [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18a1a0a446e655ef02aa7598aabe8703 columnFamilyName cf1 2023-07-12 19:10:34,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-07-12 19:10:34,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure c0e9dac5a8ee81c6ad08618940a5f514, server=jenkins-hbase20.apache.org,38353,1689189020667 in 274 msec 2023-07-12 19:10:34,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0e9dac5a8ee81c6ad08618940a5f514, ASSIGN in 447 msec 2023-07-12 19:10:34,244 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] throttle.PressureAwareThroughputController(145): c0e9dac5a8ee81c6ad08618940a5f514#cf1#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:34,244 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] throttle.PressureAwareThroughputController(145): c0e9dac5a8ee81c6ad08618940a5f514#cf2#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:34,247 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1-top 2023-07-12 19:10:34,259 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/246096b68ff2402b9dcb312a377d81c3-top 2023-07-12 19:10:34,264 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5701a846a9024ee99cb023d93ddf021f-top 2023-07-12 19:10:34,271 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5aef80d484ce4cd6b6fc892fa133a77b-top 2023-07-12 19:10:34,271 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/.tmp/cf1/bf6e2b5b98df498c9a00347c45d1dc54 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bf6e2b5b98df498c9a00347c45d1dc54 2023-07-12 19:10:34,281 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c0e9dac5a8ee81c6ad08618940a5f514/cf1 of c0e9dac5a8ee81c6ad08618940a5f514 into bf6e2b5b98df498c9a00347c45d1dc54(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:34,281 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c0e9dac5a8ee81c6ad08618940a5f514: 2023-07-12 19:10:34,281 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514., storeName=c0e9dac5a8ee81c6ad08618940a5f514/cf1, priority=9, startTime=1689189034222; duration=0sec 2023-07-12 19:10:34,282 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:34,282 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/7f49c040019b4ea8a74a717a4da2b9d9-top 2023-07-12 19:10:34,283 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/.tmp/cf2/444b30c4d071466f90b42d8930dded7c as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/444b30c4d071466f90b42d8930dded7c 2023-07-12 19:10:34,286 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bc3adf313ea64f7d90c1145883882006-top 2023-07-12 19:10:34,288 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c0e9dac5a8ee81c6ad08618940a5f514/cf2 of c0e9dac5a8ee81c6ad08618940a5f514 into 444b30c4d071466f90b42d8930dded7c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:34,288 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for c0e9dac5a8ee81c6ad08618940a5f514: 2023-07-12 19:10:34,288 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514., storeName=c0e9dac5a8ee81c6ad08618940a5f514/cf2, priority=9, startTime=1689189034223; duration=0sec 2023-07-12 19:10:34,288 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:34,292 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bd0eba394a11405c910688e18771a4cc-top 2023-07-12 19:10:34,292 INFO [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(310): Store=18a1a0a446e655ef02aa7598aabe8703/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:34,292 INFO [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:34,293 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2 2023-07-12 19:10:34,293 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2 2023-07-12 19:10:34,293 INFO [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18a1a0a446e655ef02aa7598aabe8703 columnFamilyName cf2 2023-07-12 19:10:34,310 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/0c1a752213484b179c6458186fd9482b-top 2023-07-12 19:10:34,314 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/31b19a3025614e018c92f7d6b1cd6c9a-top 2023-07-12 19:10:34,317 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/58e66d0dabaf481cb438e9ead062860d-top 2023-07-12 19:10:34,322 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/bb802536c2224693bd5ede242b652340-top 2023-07-12 19:10:34,326 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/ceb560de5aec4f6dac7ad5e9c558d976-top 2023-07-12 19:10:34,331 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/e61e803e20d24b718b6aa65468c1f60c-top 2023-07-12 19:10:34,337 DEBUG [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/f2f6fbf463104434a65a704f32849165-top 2023-07-12 19:10:34,337 INFO [StoreOpener-18a1a0a446e655ef02aa7598aabe8703-1] regionserver.HStore(310): Store=18a1a0a446e655ef02aa7598aabe8703/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:34,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:34,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:34,340 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:10:34,341 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:34,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 18a1a0a446e655ef02aa7598aabe8703; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61401389, jitterRate=-0.08504800498485565}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:10:34,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 18a1a0a446e655ef02aa7598aabe8703: 2023-07-12 19:10:34,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703., pid=33, masterSystemTime=1689189034100 2023-07-12 19:10:34,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:10:34,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:10:34,344 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:10:34,344 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:10:34,344 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703. 2023-07-12 19:10:34,344 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1912): 18a1a0a446e655ef02aa7598aabe8703/cf1 is initiating minor compaction (all files) 2023-07-12 19:10:34,344 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703. 2023-07-12 19:10:34,344 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1912): 18a1a0a446e655ef02aa7598aabe8703/cf2 is initiating minor compaction (all files) 2023-07-12 19:10:34,344 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 18a1a0a446e655ef02aa7598aabe8703/cf1 in testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703. 2023-07-12 19:10:34,344 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 18a1a0a446e655ef02aa7598aabe8703/cf2 in testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703. 2023-07-12 19:10:34,344 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5aef80d484ce4cd6b6fc892fa133a77b-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/246096b68ff2402b9dcb312a377d81c3-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/7f49c040019b4ea8a74a717a4da2b9d9-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bd0eba394a11405c910688e18771a4cc-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5701a846a9024ee99cb023d93ddf021f-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bc3adf313ea64f7d90c1145883882006-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1-top] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/.tmp, totalSize=34.9 K 2023-07-12 19:10:34,344 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/58e66d0dabaf481cb438e9ead062860d-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/0c1a752213484b179c6458186fd9482b-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/31b19a3025614e018c92f7d6b1cd6c9a-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/e61e803e20d24b718b6aa65468c1f60c-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/ceb560de5aec4f6dac7ad5e9c558d976-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/bb802536c2224693bd5ede242b652340-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/f2f6fbf463104434a65a704f32849165-top] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/.tmp, totalSize=34.9 K 2023-07-12 19:10:34,345 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703. 2023-07-12 19:10:34,346 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689189029044 2023-07-12 19:10:34,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703. 2023-07-12 19:10:34,346 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689189029044 2023-07-12 19:10:34,346 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=18a1a0a446e655ef02aa7598aabe8703, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:34,346 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689189029243 2023-07-12 19:10:34,346 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689189029243 2023-07-12 19:10:34,347 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189034346"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189034346"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189034346"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189034346"}]},"ts":"1689189034346"} 2023-07-12 19:10:34,347 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689189029458 2023-07-12 19:10:34,347 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689189029458 2023-07-12 19:10:34,347 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689189029682 2023-07-12 19:10:34,347 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689189029682 2023-07-12 19:10:34,348 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1689189029910 2023-07-12 19:10:34,348 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1689189029910 2023-07-12 19:10:34,348 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1689189030116 2023-07-12 19:10:34,348 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1689189030116 2023-07-12 19:10:34,349 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689189030320 2023-07-12 19:10:34,349 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689189030320 2023-07-12 19:10:34,351 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-07-12 19:10:34,351 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure 18a1a0a446e655ef02aa7598aabe8703, server=jenkins-hbase20.apache.org,38353,1689189020667 in 401 msec 2023-07-12 19:10:34,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-07-12 19:10:34,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18a1a0a446e655ef02aa7598aabe8703, ASSIGN in 568 msec 2023-07-12 19:10:34,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 in 3.7630 sec 2023-07-12 19:10:34,361 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] throttle.PressureAwareThroughputController(145): 18a1a0a446e655ef02aa7598aabe8703#cf1#compaction#34 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:34,363 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] throttle.PressureAwareThroughputController(145): 18a1a0a446e655ef02aa7598aabe8703#cf2#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:34,387 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/.tmp/cf1/d0bf3b65c03c4e0bb06a69853413a02c as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/d0bf3b65c03c4e0bb06a69853413a02c 2023-07-12 19:10:34,392 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/.tmp/cf2/c45acd67ce284773947d29700b7a46d4 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/c45acd67ce284773947d29700b7a46d4 2023-07-12 19:10:34,393 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 18a1a0a446e655ef02aa7598aabe8703/cf1 of 18a1a0a446e655ef02aa7598aabe8703 into d0bf3b65c03c4e0bb06a69853413a02c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:34,393 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 18a1a0a446e655ef02aa7598aabe8703: 2023-07-12 19:10:34,393 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703., storeName=18a1a0a446e655ef02aa7598aabe8703/cf1, priority=9, startTime=1689189034343; duration=0sec 2023-07-12 19:10:34,394 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:34,397 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 18a1a0a446e655ef02aa7598aabe8703/cf2 of 18a1a0a446e655ef02aa7598aabe8703 into c45acd67ce284773947d29700b7a46d4(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:34,397 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 18a1a0a446e655ef02aa7598aabe8703: 2023-07-12 19:10:34,398 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703., storeName=18a1a0a446e655ef02aa7598aabe8703/cf2, priority=9, startTime=1689189034344; duration=0sec 2023-07-12 19:10:34,398 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:34,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38353] ipc.CallRunner(144): callId: 241 service: ClientService methodName: Scan size: 143 connection: 148.251.75.209:59964 deadline: 1689189094525, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8. is not online on jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:34,651 DEBUG [Listener at localhost.localdomain/40405] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-12 19:10:34,663 INFO [Listener at localhost.localdomain/40405] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-12 19:10:34,665 DEBUG [Listener at localhost.localdomain/40405] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-12 19:10:34,685 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:10:34,687 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-07-12 19:10:34,687 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitWithoutPONR 2023-07-12 19:10:34,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:34,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-07-12 19:10:34,691 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189034691"}]},"ts":"1689189034691"} 2023-07-12 19:10:34,693 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-07-12 19:10:34,694 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-07-12 19:10:34,694 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0e9dac5a8ee81c6ad08618940a5f514, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18a1a0a446e655ef02aa7598aabe8703, UNASSIGN}] 2023-07-12 19:10:34,695 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18a1a0a446e655ef02aa7598aabe8703, UNASSIGN 2023-07-12 19:10:34,696 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=18a1a0a446e655ef02aa7598aabe8703, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:34,696 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189034696"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189034696"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189034696"}]},"ts":"1689189034696"} 2023-07-12 19:10:34,697 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 18a1a0a446e655ef02aa7598aabe8703, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:34,699 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0e9dac5a8ee81c6ad08618940a5f514, UNASSIGN 2023-07-12 19:10:34,700 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=c0e9dac5a8ee81c6ad08618940a5f514, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:34,700 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189034700"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189034700"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189034700"}]},"ts":"1689189034700"} 2023-07-12 19:10:34,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure c0e9dac5a8ee81c6ad08618940a5f514, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:34,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-07-12 19:10:34,849 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close c0e9dac5a8ee81c6ad08618940a5f514 2023-07-12 19:10:34,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing c0e9dac5a8ee81c6ad08618940a5f514, disabling compactions & flushes 2023-07-12 19:10:34,849 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514. 2023-07-12 19:10:34,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514. 2023-07-12 19:10:34,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514. after waiting 0 ms 2023-07-12 19:10:34,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514. 2023-07-12 19:10:34,862 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5aef80d484ce4cd6b6fc892fa133a77b-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/246096b68ff2402b9dcb312a377d81c3-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/7f49c040019b4ea8a74a717a4da2b9d9-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bd0eba394a11405c910688e18771a4cc-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5701a846a9024ee99cb023d93ddf021f-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bc3adf313ea64f7d90c1145883882006-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1-bottom] to archive 2023-07-12 19:10:34,863 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:10:34,865 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,866 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,868 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,869 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,870 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,872 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,873 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,874 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/58e66d0dabaf481cb438e9ead062860d-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/0c1a752213484b179c6458186fd9482b-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/31b19a3025614e018c92f7d6b1cd6c9a-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/e61e803e20d24b718b6aa65468c1f60c-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/ceb560de5aec4f6dac7ad5e9c558d976-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/bb802536c2224693bd5ede242b652340-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/f2f6fbf463104434a65a704f32849165-bottom] to archive 2023-07-12 19:10:34,875 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:10:34,877 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,878 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,879 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,880 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,881 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,883 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,884 DEBUG [StoreCloser-testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 19:10:34,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:34,888 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514. 2023-07-12 19:10:34,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for c0e9dac5a8ee81c6ad08618940a5f514: 2023-07-12 19:10:34,889 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed c0e9dac5a8ee81c6ad08618940a5f514 2023-07-12 19:10:34,889 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:34,890 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 18a1a0a446e655ef02aa7598aabe8703, disabling compactions & flushes 2023-07-12 19:10:34,890 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703. 2023-07-12 19:10:34,890 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703. 2023-07-12 19:10:34,890 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703. after waiting 0 ms 2023-07-12 19:10:34,890 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703. 2023-07-12 19:10:34,890 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=c0e9dac5a8ee81c6ad08618940a5f514, regionState=CLOSED 2023-07-12 19:10:34,890 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189034890"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189034890"}]},"ts":"1689189034890"} 2023-07-12 19:10:34,891 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5aef80d484ce4cd6b6fc892fa133a77b-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/246096b68ff2402b9dcb312a377d81c3-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/7f49c040019b4ea8a74a717a4da2b9d9-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bd0eba394a11405c910688e18771a4cc-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5701a846a9024ee99cb023d93ddf021f-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bc3adf313ea64f7d90c1145883882006-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1-top] to archive 2023-07-12 19:10:34,892 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:10:34,894 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-07-12 19:10:34,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure c0e9dac5a8ee81c6ad08618940a5f514, server=jenkins-hbase20.apache.org,38353,1689189020667 in 191 msec 2023-07-12 19:10:34,895 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0e9dac5a8ee81c6ad08618940a5f514, UNASSIGN in 200 msec 2023-07-12 19:10:34,897 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,898 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,899 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,900 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,901 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,903 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/58e66d0dabaf481cb438e9ead062860d-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/0c1a752213484b179c6458186fd9482b-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/31b19a3025614e018c92f7d6b1cd6c9a-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/e61e803e20d24b718b6aa65468c1f60c-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/ceb560de5aec4f6dac7ad5e9c558d976-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/bb802536c2224693bd5ede242b652340-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/f2f6fbf463104434a65a704f32849165-top] to archive 2023-07-12 19:10:34,904 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:10:34,905 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,906 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,908 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,909 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,910 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,911 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,912 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:34,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 19:10:34,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:34,918 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703. 2023-07-12 19:10:34,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 18a1a0a446e655ef02aa7598aabe8703: 2023-07-12 19:10:34,920 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:34,920 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=18a1a0a446e655ef02aa7598aabe8703, regionState=CLOSED 2023-07-12 19:10:34,921 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189034920"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189034920"}]},"ts":"1689189034920"} 2023-07-12 19:10:34,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-07-12 19:10:34,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 18a1a0a446e655ef02aa7598aabe8703, server=jenkins-hbase20.apache.org,38353,1689189020667 in 226 msec 2023-07-12 19:10:34,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-07-12 19:10:34,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18a1a0a446e655ef02aa7598aabe8703, UNASSIGN in 231 msec 2023-07-12 19:10:34,929 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189034929"}]},"ts":"1689189034929"} 2023-07-12 19:10:34,931 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-07-12 19:10:34,932 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-07-12 19:10:34,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 245 msec 2023-07-12 19:10:34,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-07-12 19:10:34,996 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-07-12 19:10:34,998 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitWithoutPONR 2023-07-12 19:10:35,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:35,004 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:35,005 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:35,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-07-12 19:10:35,015 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,015 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514 2023-07-12 19:10:35,015 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:35,018 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/recovered.edits] 2023-07-12 19:10:35,018 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/recovered.edits] 2023-07-12 19:10:35,018 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/recovered.edits] 2023-07-12 19:10:35,021 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/.regioninfo to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/.regioninfo 2023-07-12 19:10:35,022 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/.regioninfo to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/.regioninfo 2023-07-12 19:10:35,022 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1 2023-07-12 19:10:35,024 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/246096b68ff2402b9dcb312a377d81c3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/246096b68ff2402b9dcb312a377d81c3 2023-07-12 19:10:35,025 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,025 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf1/0c2f95eb968b4a65a4e31fac3cd3aaf1.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,025 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5701a846a9024ee99cb023d93ddf021f to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5701a846a9024ee99cb023d93ddf021f 2023-07-12 19:10:35,026 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf1/246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf1/246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,026 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf1/246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf1/246096b68ff2402b9dcb312a377d81c3.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,026 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5aef80d484ce4cd6b6fc892fa133a77b to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/5aef80d484ce4cd6b6fc892fa133a77b 2023-07-12 19:10:35,027 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf1/5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf1/5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,027 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf1/5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf1/5701a846a9024ee99cb023d93ddf021f.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,027 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/7f49c040019b4ea8a74a717a4da2b9d9 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/7f49c040019b4ea8a74a717a4da2b9d9 2023-07-12 19:10:35,028 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf1/5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf1/5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,028 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf1/5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf1/5aef80d484ce4cd6b6fc892fa133a77b.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,029 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bc3adf313ea64f7d90c1145883882006 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bc3adf313ea64f7d90c1145883882006 2023-07-12 19:10:35,030 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf1/7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf1/7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,030 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf1/7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf1/7f49c040019b4ea8a74a717a4da2b9d9.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,030 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bd0eba394a11405c910688e18771a4cc to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf1/bd0eba394a11405c910688e18771a4cc 2023-07-12 19:10:35,031 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,031 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf1/bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf1/bc3adf313ea64f7d90c1145883882006.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,032 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,033 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf1/bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf1/bd0eba394a11405c910688e18771a4cc.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,033 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/0c1a752213484b179c6458186fd9482b to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/0c1a752213484b179c6458186fd9482b 2023-07-12 19:10:35,035 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/31b19a3025614e018c92f7d6b1cd6c9a to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/31b19a3025614e018c92f7d6b1cd6c9a 2023-07-12 19:10:35,036 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf2/0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf2/0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,036 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf2/0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf2/0c1a752213484b179c6458186fd9482b.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,037 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/58e66d0dabaf481cb438e9ead062860d to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/58e66d0dabaf481cb438e9ead062860d 2023-07-12 19:10:35,037 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf2/31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf2/31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,037 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf2/31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf2/31b19a3025614e018c92f7d6b1cd6c9a.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,038 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/bb802536c2224693bd5ede242b652340 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/bb802536c2224693bd5ede242b652340 2023-07-12 19:10:35,039 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf2/58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf2/58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,039 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf2/58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf2/58e66d0dabaf481cb438e9ead062860d.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,040 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/ceb560de5aec4f6dac7ad5e9c558d976 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/ceb560de5aec4f6dac7ad5e9c558d976 2023-07-12 19:10:35,040 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf2/bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf2/bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,041 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf2/bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf2/bb802536c2224693bd5ede242b652340.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,042 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/e61e803e20d24b718b6aa65468c1f60c to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/e61e803e20d24b718b6aa65468c1f60c 2023-07-12 19:10:35,042 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf2/ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf2/ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,043 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf2/ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf2/ceb560de5aec4f6dac7ad5e9c558d976.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,043 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/f2f6fbf463104434a65a704f32849165 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/cf2/f2f6fbf463104434a65a704f32849165 2023-07-12 19:10:35,044 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf2/e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf2/e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,044 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf2/e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf2/e61e803e20d24b718b6aa65468c1f60c.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,045 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf2/f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/c0e9dac5a8ee81c6ad08618940a5f514/cf2/f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,046 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf2/f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/18a1a0a446e655ef02aa7598aabe8703/cf2/f2f6fbf463104434a65a704f32849165.d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,047 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/recovered.edits/85.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8/recovered.edits/85.seqid 2023-07-12 19:10:35,047 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/d0505bae67c35c958e3be134081a0fc8 2023-07-12 19:10:35,049 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bf6e2b5b98df498c9a00347c45d1dc54 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf1/bf6e2b5b98df498c9a00347c45d1dc54 2023-07-12 19:10:35,049 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/d0bf3b65c03c4e0bb06a69853413a02c to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf1/d0bf3b65c03c4e0bb06a69853413a02c 2023-07-12 19:10:35,051 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/444b30c4d071466f90b42d8930dded7c to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/cf2/444b30c4d071466f90b42d8930dded7c 2023-07-12 19:10:35,051 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/c45acd67ce284773947d29700b7a46d4 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/cf2/c45acd67ce284773947d29700b7a46d4 2023-07-12 19:10:35,054 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/recovered.edits/90.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514/recovered.edits/90.seqid 2023-07-12 19:10:35,054 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/recovered.edits/90.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703/recovered.edits/90.seqid 2023-07-12 19:10:35,055 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/c0e9dac5a8ee81c6ad08618940a5f514 2023-07-12 19:10:35,055 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitWithoutPONR/18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:35,055 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-07-12 19:10:35,058 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:35,060 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-07-12 19:10:35,062 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-07-12 19:10:35,064 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:35,064 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-07-12 19:10:35,064 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189035064"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:35,064 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189035064"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:35,064 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189035064"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:35,066 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-12 19:10:35,066 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d0505bae67c35c958e3be134081a0fc8, NAME => 'testSplitWithoutPONR,,1689189028536.d0505bae67c35c958e3be134081a0fc8.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c0e9dac5a8ee81c6ad08618940a5f514, NAME => 'testSplitWithoutPONR,,1689189030590.c0e9dac5a8ee81c6ad08618940a5f514.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 18a1a0a446e655ef02aa7598aabe8703, NAME => 'testSplitWithoutPONR,41,1689189030590.18a1a0a446e655ef02aa7598aabe8703.', STARTKEY => '41', ENDKEY => ''}] 2023-07-12 19:10:35,067 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-07-12 19:10:35,067 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689189035067"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:35,068 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-07-12 19:10:35,070 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:35,071 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 72 msec 2023-07-12 19:10:35,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-07-12 19:10:35,108 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-07-12 19:10:35,120 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=398 (was 393) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:51882 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:47514 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:51864 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:60850 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:60912 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:60830 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:51868 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:51858 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:60904 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:60884 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:60866 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7acefbf8-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:47520 [Waiting for operation #31] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:60852 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:51854 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:51870 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:60856 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=862 (was 839) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=204 (was 231), ProcessCount=170 (was 174), AvailableMemoryMB=5864 (was 5990) 2023-07-12 19:10:35,132 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=398, OpenFileDescriptor=862, MaxFileDescriptor=60000, SystemLoadAverage=204, ProcessCount=170, AvailableMemoryMB=5863 2023-07-12 19:10:35,133 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:10:35,134 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 19:10:35,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 19:10:35,136 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:35,138 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 19:10:35,138 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 19:10:35,140 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:35,140 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6 empty. 2023-07-12 19:10:35,141 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:35,141 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-07-12 19:10:35,151 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-07-12 19:10:35,152 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => eb3ad527ad66eb4244896aace6f290b6, NAME => 'testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp 2023-07-12 19:10:35,161 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:35,161 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing eb3ad527ad66eb4244896aace6f290b6, disabling compactions & flushes 2023-07-12 19:10:35,161 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:35,161 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:35,161 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. after waiting 0 ms 2023-07-12 19:10:35,161 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:35,161 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:35,162 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for eb3ad527ad66eb4244896aace6f290b6: 2023-07-12 19:10:35,164 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 19:10:35,165 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189035165"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189035165"}]},"ts":"1689189035165"} 2023-07-12 19:10:35,167 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 19:10:35,168 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 19:10:35,168 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189035168"}]},"ts":"1689189035168"} 2023-07-12 19:10:35,169 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-07-12 19:10:35,172 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 19:10:35,173 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 19:10:35,173 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 19:10:35,173 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 19:10:35,173 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 19:10:35,173 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN}] 2023-07-12 19:10:35,175 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:10:35,176 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,33255,1689189020635; forceNewPlan=false, retain=false 2023-07-12 19:10:35,326 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 19:10:35,329 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=eb3ad527ad66eb4244896aace6f290b6, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,329 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189035328"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189035328"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189035328"}]},"ts":"1689189035328"} 2023-07-12 19:10:35,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:10:35,489 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,490 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 19:10:35,496 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56292, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 19:10:35,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:35,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => eb3ad527ad66eb4244896aace6f290b6, NAME => 'testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.', STARTKEY => '', ENDKEY => ''} 2023-07-12 19:10:35,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:35,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:35,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:35,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:35,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:35,514 INFO [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:35,517 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1 2023-07-12 19:10:35,517 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1 2023-07-12 19:10:35,518 INFO [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eb3ad527ad66eb4244896aace6f290b6 columnFamilyName cf1 2023-07-12 19:10:35,519 INFO [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(310): Store=eb3ad527ad66eb4244896aace6f290b6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:35,520 INFO [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:35,522 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2 2023-07-12 19:10:35,523 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2 2023-07-12 19:10:35,523 INFO [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eb3ad527ad66eb4244896aace6f290b6 columnFamilyName cf2 2023-07-12 19:10:35,524 INFO [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(310): Store=eb3ad527ad66eb4244896aace6f290b6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:35,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:35,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:35,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:10:35,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:35,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 19:10:35,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened eb3ad527ad66eb4244896aace6f290b6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58953931, jitterRate=-0.12151797115802765}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:10:35,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for eb3ad527ad66eb4244896aace6f290b6: 2023-07-12 19:10:35,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6., pid=43, masterSystemTime=1689189035489 2023-07-12 19:10:35,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:35,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:35,539 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=eb3ad527ad66eb4244896aace6f290b6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,539 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189035539"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189035539"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189035539"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189035539"}]},"ts":"1689189035539"} 2023-07-12 19:10:35,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-07-12 19:10:35,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 in 207 msec 2023-07-12 19:10:35,544 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-07-12 19:10:35,544 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN in 370 msec 2023-07-12 19:10:35,545 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 19:10:35,545 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189035545"}]},"ts":"1689189035545"} 2023-07-12 19:10:35,547 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-07-12 19:10:35,549 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 19:10:35,550 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 414 msec 2023-07-12 19:10:35,642 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 19:10:35,649 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56298, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 19:10:35,659 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:35,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-12 19:10:35,660 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-12 19:10:35,660 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:35,660 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-12 19:10:35,660 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 19:10:35,661 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,661 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,662 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:35,662 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:35,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:35,662 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:35,662 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:35,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:35,662 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,662 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:35,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:35,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:35,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:35,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,662 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,662 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:35,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,663 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:35,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,663 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:35,663 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:35,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:35,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,663 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:35,663 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:35,663 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:35,663 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:35,663 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:35,663 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:35,663 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:35,663 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:35,663 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:35,663 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:35,663 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:35,663 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:35,663 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:35,663 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:35,663 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:35,663 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:35,663 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:35,664 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:35,664 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:35,664 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:35,664 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. started... 2023-07-12 19:10:35,664 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb3ad527ad66eb4244896aace6f290b6 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 19:10:35,665 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,665 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,665 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,665 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:35,665 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,665 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:35,678 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf1/203c1d7924984b6f86515610cab31aaa 2023-07-12 19:10:35,697 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf2/5c983e2c37cd4ab99fef58551f58b66e 2023-07-12 19:10:35,701 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf1/203c1d7924984b6f86515610cab31aaa as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/203c1d7924984b6f86515610cab31aaa 2023-07-12 19:10:35,706 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/203c1d7924984b6f86515610cab31aaa, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:10:35,707 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf2/5c983e2c37cd4ab99fef58551f58b66e as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/5c983e2c37cd4ab99fef58551f58b66e 2023-07-12 19:10:35,712 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/5c983e2c37cd4ab99fef58551f58b66e, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:10:35,713 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for eb3ad527ad66eb4244896aace6f290b6 in 49ms, sequenceid=6, compaction requested=false 2023-07-12 19:10:35,713 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-07-12 19:10:35,713 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb3ad527ad66eb4244896aace6f290b6: 2023-07-12 19:10:35,713 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:35,713 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:35,713 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:35,714 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,714 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:35,714 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:35,715 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,715 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:35,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:35,715 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,715 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:35,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:35,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:35,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:35,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:35,717 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-12 19:10:35,717 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 19:10:35,717 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6243c38e[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:35,717 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,718 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,718 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,718 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,718 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:35,718 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:35,718 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,718 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:35,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,718 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:35,718 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:35,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,718 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:35,718 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:35,718 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:35,718 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:35,718 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:35,719 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:35,719 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:35,719 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:35,719 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,719 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:35,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:35,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:35,719 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:35,719 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:35,719 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:35,719 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:35,720 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:35,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:35,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:35,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:35,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:35,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:10:35,723 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 19:10:35,723 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-12 19:10:35,723 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:35,723 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:35,723 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-12 19:10:35,723 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:35,724 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,724 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,724 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,724 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:35,724 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:35,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,724 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,725 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,725 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:35,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:35,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,725 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:35,726 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:35,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:35,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,727 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,727 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:35,727 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:35,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,729 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,729 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:35,729 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,729 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,729 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,729 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,729 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,729 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:35,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:35,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:35,729 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:35,729 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-12 19:10:35,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:35,729 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:35,729 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:35,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:35,730 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:35,730 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,730 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:35,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:35,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:35,730 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:35,730 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:35,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:35,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:35,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:35,730 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,730 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:35,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:35,730 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:35,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:35,830 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:35,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:35,846 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:35,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:35,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-12 19:10:35,848 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-12 19:10:35,848 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:35,848 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-12 19:10:35,848 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 19:10:35,849 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,849 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,850 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:35,850 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:35,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:35,850 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:35,850 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:35,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:35,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:35,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:35,850 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,850 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:35,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:35,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,851 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,851 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:35,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,851 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:35,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,851 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:35,851 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:35,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:35,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,851 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:35,851 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 19:10:35,851 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 19:10:35,851 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:35,851 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:35,851 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:35,852 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:35,852 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:35,852 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 19:10:35,852 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:35,852 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:35,852 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:35,852 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:35,852 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:35,852 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:35,852 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:35,852 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:35,852 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:35,852 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:35,852 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:35,852 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:35,852 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:35,852 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. started... 2023-07-12 19:10:35,852 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:35,853 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,853 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb3ad527ad66eb4244896aace6f290b6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:35,853 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,853 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:35,853 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,854 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,854 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:35,864 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf1/c39444aa379d411cabe6ffdd436460b5 2023-07-12 19:10:35,876 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf2/c91903d0557f49259e4dc3e82d24e7cd 2023-07-12 19:10:35,880 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf1/c39444aa379d411cabe6ffdd436460b5 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/c39444aa379d411cabe6ffdd436460b5 2023-07-12 19:10:35,885 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/c39444aa379d411cabe6ffdd436460b5, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:10:35,885 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf2/c91903d0557f49259e4dc3e82d24e7cd as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/c91903d0557f49259e4dc3e82d24e7cd 2023-07-12 19:10:35,891 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/c91903d0557f49259e4dc3e82d24e7cd, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:10:35,892 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for eb3ad527ad66eb4244896aace6f290b6 in 39ms, sequenceid=19, compaction requested=false 2023-07-12 19:10:35,892 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb3ad527ad66eb4244896aace6f290b6: 2023-07-12 19:10:35,892 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:35,892 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:35,892 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:35,892 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,892 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:35,892 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:35,893 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,893 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:35,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:35,894 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,894 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:35,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:35,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:35,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:35,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:35,896 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-12 19:10:35,896 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e1a7108[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:35,896 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 19:10:35,896 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,897 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,897 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,897 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,897 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,897 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:35,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,897 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:35,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,897 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:35,897 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:35,897 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:35,897 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:35,897 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:35,897 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:35,897 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:35,897 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:35,899 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:35,899 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,899 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:35,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:35,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:35,899 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:35,899 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:35,899 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:35,899 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:35,899 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:35,899 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:35,899 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:35,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:35,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:35,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:35,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:35,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:35,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:10:35,903 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 19:10:35,903 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-12 19:10:35,903 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:35,903 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:35,903 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-12 19:10:35,903 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:35,904 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,904 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,904 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:35,904 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,904 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,904 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:35,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:35,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:35,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:35,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,904 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:35,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:35,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:35,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:35,907 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:35,907 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:35,907 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:35,909 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,909 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,909 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,911 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,911 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:35,911 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:35,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:35,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:35,911 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:35,911 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:35,911 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,911 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-12 19:10:35,911 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:35,911 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,911 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:35,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:35,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:35,911 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:35,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:35,911 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:35,912 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:35,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:35,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:35,911 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,912 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:35,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:35,912 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:35,912 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:35,912 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,912 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:35,912 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:35,912 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:35,912 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,012 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:36,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:36,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:36,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:36,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-12 19:10:36,036 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-12 19:10:36,036 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,036 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-12 19:10:36,036 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 19:10:36,036 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,037 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,037 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,037 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,037 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,037 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,038 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,038 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:36,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,038 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,038 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,039 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:36,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,039 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:36,039 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 19:10:36,039 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:36,039 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:36,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:36,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,039 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:36,039 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:36,039 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:36,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,039 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:36,039 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:36,039 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:36,040 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 19:10:36,040 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:36,040 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:36,040 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:36,040 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,040 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. started... 2023-07-12 19:10:36,040 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:36,040 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:36,040 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:36,040 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb3ad527ad66eb4244896aace6f290b6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:36,041 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,042 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,042 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:36,042 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 19:10:36,043 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:36,043 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:36,043 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,043 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:36,043 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:36,043 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:36,044 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,044 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,044 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:36,054 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf1/9d0b63720acc4af395b05a9a5c647ab8 2023-07-12 19:10:36,069 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf2/aaac648f7ddd41fab2a4c1523eee9bab 2023-07-12 19:10:36,075 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf1/9d0b63720acc4af395b05a9a5c647ab8 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9d0b63720acc4af395b05a9a5c647ab8 2023-07-12 19:10:36,080 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9d0b63720acc4af395b05a9a5c647ab8, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:10:36,081 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf2/aaac648f7ddd41fab2a4c1523eee9bab as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/aaac648f7ddd41fab2a4c1523eee9bab 2023-07-12 19:10:36,086 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/aaac648f7ddd41fab2a4c1523eee9bab, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:10:36,087 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for eb3ad527ad66eb4244896aace6f290b6 in 47ms, sequenceid=32, compaction requested=true 2023-07-12 19:10:36,087 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb3ad527ad66eb4244896aace6f290b6: 2023-07-12 19:10:36,087 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:36,087 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:36,087 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:36,087 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,087 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:36,087 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:36,088 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,088 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:36,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:36,089 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,089 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:36,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:36,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:36,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:36,090 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-12 19:10:36,090 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54370751[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:36,090 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 19:10:36,090 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,091 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,091 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,091 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,091 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,091 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:36,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,091 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:36,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,092 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:36,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,092 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:36,092 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:36,092 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:36,092 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:36,092 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:36,092 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:36,092 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:36,093 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:36,093 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,093 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:36,093 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:36,093 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,093 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:36,093 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,093 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:36,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,093 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:36,093 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:36,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:36,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:36,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:36,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:36,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:10:36,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 19:10:36,097 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-12 19:10:36,097 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:36,097 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:36,097 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-12 19:10:36,097 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:36,103 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,103 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,103 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:36,103 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,103 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:36,103 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,103 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,103 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:36,104 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,105 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:36,106 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,106 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,107 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,112 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,112 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,112 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,112 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-12 19:10:36,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:36,112 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,112 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,112 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,113 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,113 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:36,112 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,112 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,113 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:36,113 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,113 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,113 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,113 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,113 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,114 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,114 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,114 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,213 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:36,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:36,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:36,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:36,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-12 19:10:36,235 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-12 19:10:36,235 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,235 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-12 19:10:36,235 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 19:10:36,235 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,235 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,236 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,236 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,236 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,236 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,237 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,237 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:36,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:36,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,238 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:36,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:36,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,238 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:36,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:36,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,238 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 19:10:36,238 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 19:10:36,238 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:36,238 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:36,238 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:36,238 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:36,238 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:36,238 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 19:10:36,238 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,238 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,238 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:36,238 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:36,238 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:36,238 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:36,238 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:36,238 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,238 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:36,238 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:36,238 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:36,238 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:36,238 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:36,239 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:36,239 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:36,239 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. started... 2023-07-12 19:10:36,239 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb3ad527ad66eb4244896aace6f290b6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:36,239 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,239 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,239 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,240 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:36,240 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,240 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:36,248 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf1/9aee5e79a11b4886b6851b1477a76e2e 2023-07-12 19:10:36,260 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf2/57838bef137f4697a653f6dcdb94173a 2023-07-12 19:10:36,265 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf1/9aee5e79a11b4886b6851b1477a76e2e as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9aee5e79a11b4886b6851b1477a76e2e 2023-07-12 19:10:36,270 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9aee5e79a11b4886b6851b1477a76e2e, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:10:36,271 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf2/57838bef137f4697a653f6dcdb94173a as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/57838bef137f4697a653f6dcdb94173a 2023-07-12 19:10:36,275 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/57838bef137f4697a653f6dcdb94173a, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:10:36,276 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for eb3ad527ad66eb4244896aace6f290b6 in 37ms, sequenceid=45, compaction requested=true 2023-07-12 19:10:36,277 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb3ad527ad66eb4244896aace6f290b6: 2023-07-12 19:10:36,277 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:36,277 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:36,277 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:36,277 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,277 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:36,277 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:36,278 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,278 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:36,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:36,278 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,278 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:36,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:36,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:36,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:36,280 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-12 19:10:36,280 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3021e804[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:36,280 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 19:10:36,280 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,281 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,281 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,281 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,281 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,281 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:36,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,281 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:36,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,281 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:36,282 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:36,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,281 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:36,282 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:36,282 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:36,282 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:36,282 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:36,282 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:36,282 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,282 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:36,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:36,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:36,282 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,283 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:36,283 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:36,283 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:36,283 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,283 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,283 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:36,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:36,283 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:36,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:36,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:36,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:10:36,285 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 19:10:36,285 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-12 19:10:36,285 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:36,285 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:36,285 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-12 19:10:36,285 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:36,286 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,286 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,286 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,286 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,286 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,287 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,286 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,287 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:36,287 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,287 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:36,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:36,289 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,289 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,289 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,294 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,294 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,294 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,294 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,294 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,295 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,295 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,294 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,294 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,295 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,295 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-12 19:10:36,295 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:36,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,295 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,295 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,295 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:36,295 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:36,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,295 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,295 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,296 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,296 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,296 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,296 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,395 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:36,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:36,426 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:36,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:36,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-12 19:10:36,428 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-12 19:10:36,428 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,428 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-12 19:10:36,428 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 19:10:36,428 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,428 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,429 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,429 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,429 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,429 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,430 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,430 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,430 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,430 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:36,430 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:36,430 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:36,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,431 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:36,431 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:36,431 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 19:10:36,431 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:36,431 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 19:10:36,431 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:36,431 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:36,431 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,431 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:36,431 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:36,431 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:36,431 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,431 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:36,431 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:36,431 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:36,431 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:36,432 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,432 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,432 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,432 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:36,432 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,432 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:36,434 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 19:10:36,434 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:36,435 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:36,435 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,435 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:36,435 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:36,435 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:36,435 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:36,435 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. started... 2023-07-12 19:10:36,435 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb3ad527ad66eb4244896aace6f290b6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:36,447 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf1/cf0b5ff74e784f6884f29717809791bd 2023-07-12 19:10:36,459 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf2/e9b8e07648354a4c93beed7b7f2933c6 2023-07-12 19:10:36,463 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf1/cf0b5ff74e784f6884f29717809791bd as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/cf0b5ff74e784f6884f29717809791bd 2023-07-12 19:10:36,468 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/cf0b5ff74e784f6884f29717809791bd, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:10:36,469 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf2/e9b8e07648354a4c93beed7b7f2933c6 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/e9b8e07648354a4c93beed7b7f2933c6 2023-07-12 19:10:36,474 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/e9b8e07648354a4c93beed7b7f2933c6, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:10:36,475 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for eb3ad527ad66eb4244896aace6f290b6 in 40ms, sequenceid=58, compaction requested=true 2023-07-12 19:10:36,476 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb3ad527ad66eb4244896aace6f290b6: 2023-07-12 19:10:36,476 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:36,476 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:36,476 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:36,476 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,476 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:36,476 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:36,477 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,477 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:36,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:36,478 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,478 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:36,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:36,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:36,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:36,481 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-12 19:10:36,481 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c82023d[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:36,481 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 19:10:36,481 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,481 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,481 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,481 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,482 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:36,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,482 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,482 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:36,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,482 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:36,482 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:36,482 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:36,482 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:36,482 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:36,482 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:36,482 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:36,482 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:36,484 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:36,484 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,484 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:36,485 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:36,485 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,485 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,485 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:36,485 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:36,485 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,485 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:36,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:10:36,490 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 19:10:36,490 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-12 19:10:36,490 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:36,490 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:36,490 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-12 19:10:36,490 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:36,495 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,495 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:36,495 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,495 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,496 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,495 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,495 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,496 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:36,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,497 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,497 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:36,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,500 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:36,501 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,501 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,519 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,519 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,519 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,519 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,519 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,519 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,519 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,519 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,519 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,519 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,519 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-12 19:10:36,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:36,519 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,519 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,520 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,520 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,519 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,520 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:36,520 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,520 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,520 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:36,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,621 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:36,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:36,641 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:36,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:36,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-12 19:10:36,643 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-12 19:10:36,643 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,643 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-12 19:10:36,643 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 19:10:36,643 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,643 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,644 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,644 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,644 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,644 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,644 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,644 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:36,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,645 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,645 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,645 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:36,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:36,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,645 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:36,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:36,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,645 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:36,645 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 19:10:36,645 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 19:10:36,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:36,646 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 19:10:36,646 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:36,646 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:36,646 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 19:10:36,646 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:36,646 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,646 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:36,646 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:36,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 19:10:36,646 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:36,646 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:36,646 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 19:10:36,646 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:36,646 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:36,646 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:36,646 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:10:36,646 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. started... 2023-07-12 19:10:36,646 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:36,646 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:36,646 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:36,646 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb3ad527ad66eb4244896aace6f290b6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:36,647 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,648 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,648 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,648 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:36,648 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,648 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:36,660 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf1/df375e2ea3414112949933e0590aacb1 2023-07-12 19:10:36,675 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf2/8aa2855ba2bf41818142629c6abeff33 2023-07-12 19:10:36,681 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf1/df375e2ea3414112949933e0590aacb1 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/df375e2ea3414112949933e0590aacb1 2023-07-12 19:10:36,687 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/df375e2ea3414112949933e0590aacb1, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:10:36,688 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf2/8aa2855ba2bf41818142629c6abeff33 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/8aa2855ba2bf41818142629c6abeff33 2023-07-12 19:10:36,693 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/8aa2855ba2bf41818142629c6abeff33, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:10:36,694 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for eb3ad527ad66eb4244896aace6f290b6 in 48ms, sequenceid=71, compaction requested=true 2023-07-12 19:10:36,694 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb3ad527ad66eb4244896aace6f290b6: 2023-07-12 19:10:36,694 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:36,694 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:36,694 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:36,694 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,694 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 19:10:36,694 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 19:10:36,696 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,696 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:36,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:36,696 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,696 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:36,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:36,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:36,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:36,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-12 19:10:36,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@510db894[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:36,699 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 19:10:36,699 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,700 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,700 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,700 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,701 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:36,701 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:36,701 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:36,701 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,701 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:36,701 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:36,701 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:36,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,701 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:36,701 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:10:36,701 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 19:10:36,701 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:36,702 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:36,702 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,702 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:36,703 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:36,703 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,703 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:36,703 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 19:10:36,703 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,703 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:36,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:36,703 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 19:10:36,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:36,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:36,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:36,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:10:36,708 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 19:10:36,708 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-12 19:10:36,708 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:36,708 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:36,708 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-12 19:10:36,708 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:36,719 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,719 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:36,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:36,719 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,719 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,719 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,720 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:36,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,719 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,720 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:36,721 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,722 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:36,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 19:10:36,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,726 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,726 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,727 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,731 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,731 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,731 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,731 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:36,731 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,731 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-12 19:10:36,731 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:36,731 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,731 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:36,731 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,732 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,731 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,731 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:36,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:36,732 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 19:10:36,732 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:36,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,732 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:36,732 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:36,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:36,732 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,732 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:36,733 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:36,733 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 19:10:36,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:36,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:36,733 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 19:10:36,832 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:36,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:37,103 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-07-12 19:10:37,106 DEBUG [Listener at localhost.localdomain/40405] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:37,108 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,110 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,111 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,111 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,112 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:37,112 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:37,113 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:37,359 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,359 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:37,359 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:37,360 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:37,360 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-07-12 19:10:37,361 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:37,361 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:37,361 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:37,362 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:37,364 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 2 msec 2023-07-12 19:10:37,369 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-07-12 19:10:37,369 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:37,371 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:37,371 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:37,371 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:37,371 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:37,372 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:37,372 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:37,372 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:37,372 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:37,373 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:37,373 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:37,373 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:37,373 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:37,373 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,374 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:37,374 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,374 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,374 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,374 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-12 19:10:37,375 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:37,375 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:37,376 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189067376 2023-07-12 19:10:37,377 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:37,377 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,380 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:37,380 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:37,380 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:37,380 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189097380 2023-07-12 19:10:37,381 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189157381 2023-07-12 19:10:37,381 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 19:10:37,381 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:37,383 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:37,383 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:37,383 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:37,631 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,631 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:37,631 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:37,632 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:37,632 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:37,632 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:37,632 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:37,632 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:37,633 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:37,634 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:37,636 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-12 19:10:37,637 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:37,637 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:37,638 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:37,638 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:37,639 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:37,640 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:37,640 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:37,641 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:37,641 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:37,642 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:37,642 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:37,643 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:37,643 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:37,643 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,643 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:37,643 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,643 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,643 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,643 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-07-12 19:10:37,644 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:37,645 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:37,645 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189067645 2023-07-12 19:10:37,647 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,647 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:37,649 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:37,650 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:37,650 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:37,650 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189097650 2023-07-12 19:10:37,650 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189157650 2023-07-12 19:10:37,650 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:10:37,650 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:37,653 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN}] 2023-07-12 19:10:37,653 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:37,653 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,653 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:37,653 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:37,900 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,901 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:37,901 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:37,901 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:37,902 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:37,902 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:37,902 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:37,903 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:37,903 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:37,905 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:37,909 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-12 19:10:37,909 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:37,910 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:37,911 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:37,911 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:37,912 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:37,913 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:37,913 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:37,914 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:37,914 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:37,915 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:37,915 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:37,915 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:37,916 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:37,916 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,916 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:37,916 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,916 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,916 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,916 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-07-12 19:10:37,917 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:37,918 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:37,918 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189067918 2023-07-12 19:10:37,919 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:37,919 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:37,922 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:37,922 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:37,922 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:37,922 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189097922 2023-07-12 19:10:37,923 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189157923 2023-07-12 19:10:37,923 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:10:37,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN}] 2023-07-12 19:10:37,923 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:37,926 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:37,926 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:37,927 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189037926"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189037926"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189037926"}]},"ts":"1689189037926"} 2023-07-12 19:10:37,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:10:37,928 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:37,928 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:37,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:37,928 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:38,173 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,173 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:38,173 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:38,173 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:38,174 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:38,174 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:38,174 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:38,174 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:38,175 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:38,176 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:38,177 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 19:10:38,177 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-12 19:10:38,178 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:38,178 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:38,179 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:38,179 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:38,179 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:38,179 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:38,180 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:38,180 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:38,180 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:38,181 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:38,181 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:38,181 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:38,181 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:38,182 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,182 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:38,182 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:38,182 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,182 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,183 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,183 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:38,183 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:38,183 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 19:10:38,184 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:38,184 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:38,184 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6 to restore RIT 2023-07-12 19:10:38,185 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189068185 2023-07-12 19:10:38,185 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:38,186 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:38,188 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:38,189 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSING, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:38,189 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:38,189 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189098189 2023-07-12 19:10:38,189 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189158189 2023-07-12 19:10:38,189 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 19:10:38,189 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:38,189 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189038189"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189038189"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189038189"}]},"ts":"1689189038189"} 2023-07-12 19:10:38,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:10:38,191 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:38,344 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:38,344 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing eb3ad527ad66eb4244896aace6f290b6, disabling compactions & flushes 2023-07-12 19:10:38,345 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:38,345 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:38,345 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. after waiting 0 ms 2023-07-12 19:10:38,345 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:38,345 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing eb3ad527ad66eb4244896aace6f290b6 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-12 19:10:38,354 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf1/8bfbc793cd9845c29206d5bfdf50ca7a 2023-07-12 19:10:38,369 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf2/f86bd59364af4013a8141384d60fa08b 2023-07-12 19:10:38,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf1/8bfbc793cd9845c29206d5bfdf50ca7a as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/8bfbc793cd9845c29206d5bfdf50ca7a 2023-07-12 19:10:38,379 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/8bfbc793cd9845c29206d5bfdf50ca7a, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 19:10:38,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf2/f86bd59364af4013a8141384d60fa08b as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/f86bd59364af4013a8141384d60fa08b 2023-07-12 19:10:38,384 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/f86bd59364af4013a8141384d60fa08b, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 19:10:38,384 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for eb3ad527ad66eb4244896aace6f290b6 in 39ms, sequenceid=82, compaction requested=true 2023-07-12 19:10:38,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-12 19:10:38,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:38,389 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:38,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for eb3ad527ad66eb4244896aace6f290b6: 2023-07-12 19:10:38,391 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:38,392 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED 2023-07-12 19:10:38,392 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189038392"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189038392"}]},"ts":"1689189038392"} 2023-07-12 19:10:38,393 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:38,393 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:38,393 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:38,393 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:38,439 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,439 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:38,439 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:38,440 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:38,440 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:38,440 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:38,440 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:38,440 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:38,440 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:38,441 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 19:10:38,442 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 19:10:38,442 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-12 19:10:38,442 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-07-12 19:10:38,442 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:38,443 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:38,443 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:38,443 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:38,443 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:38,444 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:38,444 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:38,444 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:38,444 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:38,445 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:38,445 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:38,445 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:38,445 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:38,446 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,446 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:38,446 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:38,447 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:38,447 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,447 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,447 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,447 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:38,447 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:38,447 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:38,447 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 19:10:38,448 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:38,449 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:38,449 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6 to restore RIT 2023-07-12 19:10:38,449 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189068449 2023-07-12 19:10:38,450 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:38,450 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:38,452 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:38,452 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:38,453 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:38,453 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189098453 2023-07-12 19:10:38,453 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189158453 2023-07-12 19:10:38,453 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 19:10:38,453 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:38,453 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189038453"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189038453"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189038453"}]},"ts":"1689189038453"} 2023-07-12 19:10:38,455 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:38,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-07-12 19:10:38,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 in 264 msec 2023-07-12 19:10:38,458 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:38,458 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:38,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:38,459 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:38,703 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,704 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:38,704 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:38,704 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:38,705 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:38,705 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:38,705 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:38,706 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:38,706 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:38,708 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:38,709 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 19:10:38,709 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-12 19:10:38,709 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:10:38,710 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:38,711 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:38,711 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:38,711 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:38,712 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:38,713 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:38,713 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:38,713 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:38,714 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:38,715 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:38,715 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:38,715 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:38,716 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:38,717 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,718 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:38,718 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:38,718 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:38,718 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,718 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,719 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,719 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:38,719 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:38,719 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-07-12 19:10:38,721 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:38,722 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:38,722 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6 to restore RIT 2023-07-12 19:10:38,722 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189068722 2023-07-12 19:10:38,723 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:38,723 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:38,725 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:38,725 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:38,725 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:38,725 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189098725 2023-07-12 19:10:38,725 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189158725 2023-07-12 19:10:38,726 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:10:38,726 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:38,727 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:38,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-07-12 19:10:38,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN in 804 msec 2023-07-12 19:10:38,736 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:38,736 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,736 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:38,736 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:38,976 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,976 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:38,976 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:38,976 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:38,976 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:38,977 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:38,977 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:38,977 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:38,977 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:38,978 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:38,979 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 19:10:38,980 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:10:38,980 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:10:38,980 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:38,981 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:38,981 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:38,981 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:38,982 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:38,983 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:38,983 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:38,983 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:38,984 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:38,984 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:38,985 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:38,985 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:38,985 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:38,986 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,986 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:38,986 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:38,987 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:38,987 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:38,987 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:38,987 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-12 19:10:38,987 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:38,988 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:38,988 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189068988 2023-07-12 19:10:38,988 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:38,990 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,992 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:38,992 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:38,992 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:38,992 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189098992 2023-07-12 19:10:38,992 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189158992 2023-07-12 19:10:38,992 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:10:38,992 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,993 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,993 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:38,997 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:39,005 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=eb3ad527ad66eb4244896aace6f290b6, threads=14 2023-07-12 19:10:39,006 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/57838bef137f4697a653f6dcdb94173a for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,006 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/5c983e2c37cd4ab99fef58551f58b66e for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,006 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/8aa2855ba2bf41818142629c6abeff33 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,006 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/aaac648f7ddd41fab2a4c1523eee9bab for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,006 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/c91903d0557f49259e4dc3e82d24e7cd for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,006 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/e9b8e07648354a4c93beed7b7f2933c6 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,006 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/f86bd59364af4013a8141384d60fa08b for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,007 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/203c1d7924984b6f86515610cab31aaa for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,007 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/8bfbc793cd9845c29206d5bfdf50ca7a for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,007 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9aee5e79a11b4886b6851b1477a76e2e for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,007 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9d0b63720acc4af395b05a9a5c647ab8 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,007 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/c39444aa379d411cabe6ffdd436460b5 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,007 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/cf0b5ff74e784f6884f29717809791bd for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,007 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/df375e2ea3414112949933e0590aacb1 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,176 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/8aa2855ba2bf41818142629c6abeff33 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,194 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/57838bef137f4697a653f6dcdb94173a for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,195 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/203c1d7924984b6f86515610cab31aaa for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,199 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9aee5e79a11b4886b6851b1477a76e2e for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,207 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/5c983e2c37cd4ab99fef58551f58b66e for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,211 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/c91903d0557f49259e4dc3e82d24e7cd for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,223 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/aaac648f7ddd41fab2a4c1523eee9bab for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,226 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/f86bd59364af4013a8141384d60fa08b for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,230 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/8bfbc793cd9845c29206d5bfdf50ca7a for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,231 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/cf0b5ff74e784f6884f29717809791bd for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,231 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/df375e2ea3414112949933e0590aacb1 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,233 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9d0b63720acc4af395b05a9a5c647ab8 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,234 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/e9b8e07648354a4c93beed7b7f2933c6 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,234 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/c39444aa379d411cabe6ffdd436460b5 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,234 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region eb3ad527ad66eb4244896aace6f290b6 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 19:10:39,274 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:39,274 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:39,274 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:39,274 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:39,493 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:39,493 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:39,493 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:39,493 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:39,496 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:39,496 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:39,496 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:39,496 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:39,496 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:39,497 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 19:10:39,498 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-12 19:10:39,498 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:10:39,498 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:10:39,498 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:39,499 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:39,499 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:39,499 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:39,500 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:39,500 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:39,500 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:39,500 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:39,501 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:39,501 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:39,501 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:39,502 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:39,502 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:39,503 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:39,503 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:39,503 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:39,503 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:39,503 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:39,503 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:39,503 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:39,504 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:39,504 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:39,504 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 19:10:39,505 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:39,505 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:39,506 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189069506 2023-07-12 19:10:39,507 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:39,507 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:39,509 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:39,509 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:39,509 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:39,510 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189099510 2023-07-12 19:10:39,510 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189159510 2023-07-12 19:10:39,510 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 19:10:39,518 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=eb3ad527ad66eb4244896aace6f290b6, threads=14 2023-07-12 19:10:39,519 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/57838bef137f4697a653f6dcdb94173a for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,519 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/5c983e2c37cd4ab99fef58551f58b66e for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,519 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/8aa2855ba2bf41818142629c6abeff33 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,520 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/aaac648f7ddd41fab2a4c1523eee9bab for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,520 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/c91903d0557f49259e4dc3e82d24e7cd for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,520 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/e9b8e07648354a4c93beed7b7f2933c6 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,520 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/f86bd59364af4013a8141384d60fa08b for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,520 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/203c1d7924984b6f86515610cab31aaa for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,520 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/8bfbc793cd9845c29206d5bfdf50ca7a for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,521 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9aee5e79a11b4886b6851b1477a76e2e for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,521 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9d0b63720acc4af395b05a9a5c647ab8 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,521 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/cf0b5ff74e784f6884f29717809791bd for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,521 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/c39444aa379d411cabe6ffdd436460b5 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,521 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/df375e2ea3414112949933e0590aacb1 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,594 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9aee5e79a11b4886b6851b1477a76e2e for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,595 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/8bfbc793cd9845c29206d5bfdf50ca7a for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,600 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/203c1d7924984b6f86515610cab31aaa for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,607 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 19:10:39,610 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/57838bef137f4697a653f6dcdb94173a for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,617 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/8aa2855ba2bf41818142629c6abeff33 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,623 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/f86bd59364af4013a8141384d60fa08b for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,655 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/c91903d0557f49259e4dc3e82d24e7cd for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,656 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/c39444aa379d411cabe6ffdd436460b5 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,659 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/aaac648f7ddd41fab2a4c1523eee9bab for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,662 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9d0b63720acc4af395b05a9a5c647ab8 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,664 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/df375e2ea3414112949933e0590aacb1 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,683 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/cf0b5ff74e784f6884f29717809791bd for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,686 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/5c983e2c37cd4ab99fef58551f58b66e for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,686 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/e9b8e07648354a4c93beed7b7f2933c6 for region: eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:39,686 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region eb3ad527ad66eb4244896aace6f290b6 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 19:10:39,721 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:39,727 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/7f35573e33b5dfbc24a5253b928fa5ba/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 19:10:39,729 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/072cbd765c3d2a7e7a54842c70ef7f0e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 19:10:39,729 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:39,729 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:39,729 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:39,729 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:39,760 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:39,760 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:39,760 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:39,760 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:39,760 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:39,760 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:39,761 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:39,761 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:39,761 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:39,761 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 19:10:39,762 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-12 19:10:39,762 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:10:39,762 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:10:39,763 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:39,763 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:39,763 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:39,763 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:39,764 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:39,764 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:39,764 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:39,764 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:39,765 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:39,765 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:39,765 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:39,765 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:39,765 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:39,766 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:39,766 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:39,766 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:39,768 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:39,768 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:39,768 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:39,768 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:39,768 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:39,768 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:39,768 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 19:10:39,769 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:39,770 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:39,770 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189069770 2023-07-12 19:10:39,771 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:39,771 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:39,774 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:39,774 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:39,774 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:39,774 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189099774 2023-07-12 19:10:39,774 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189159774 2023-07-12 19:10:39,774 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:10:39,777 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:39,781 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:39,781 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:39,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:39,781 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:40,026 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,026 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:40,026 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:40,026 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:40,027 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:40,027 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:40,027 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:40,027 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:40,028 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:40,029 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:40,032 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-12 19:10:40,032 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:10:40,032 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:10:40,033 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:40,034 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:40,034 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:40,034 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:40,035 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:40,035 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:40,035 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:40,035 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:40,035 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:40,036 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:40,036 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:40,036 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:40,036 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:40,037 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,037 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:40,037 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:40,037 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:40,037 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,037 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,037 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,037 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:40,037 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:40,037 DEBUG [Listener at localhost.localdomain/40405] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,037 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-12 19:10:40,038 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:40,038 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:40,039 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189070039 2023-07-12 19:10:40,039 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:40,042 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,044 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:40,044 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:40,044 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:40,044 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189100044 2023-07-12 19:10:40,044 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189160044 2023-07-12 19:10:40,044 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 19:10:40,045 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:40,060 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:40,060 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-12 19:10:40,060 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:40,060 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:40,297 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,297 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:40,298 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:40,298 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:40,298 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:40,299 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:40,299 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:40,299 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:40,300 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:40,302 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:40,303 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-07-12 19:10:40,303 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:10:40,304 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:10:40,304 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:40,304 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:40,304 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:40,305 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:40,305 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:40,305 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:40,305 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:40,305 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:40,305 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:40,306 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:40,306 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:40,306 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:40,307 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:40,307 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,307 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:40,307 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:40,308 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:40,308 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,308 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,308 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,308 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:40,308 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:40,308 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 19:10:40,309 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:40,309 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:40,310 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189070310 2023-07-12 19:10:40,311 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:40,312 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,314 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:40,315 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:40,315 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:40,315 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189100315 2023-07-12 19:10:40,315 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189160315 2023-07-12 19:10:40,315 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:10:40,315 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:40,317 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:40,317 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-12 19:10:40,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:40,317 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:40,567 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,567 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:40,567 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:40,568 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:40,568 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:40,569 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:40,569 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:40,569 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:40,569 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:40,571 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:40,574 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-12 19:10:40,574 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:10:40,575 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:10:40,576 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:40,577 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:40,578 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:40,578 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:40,579 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:40,579 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:40,579 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:40,579 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:40,580 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:40,580 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:40,580 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:40,580 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:40,580 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:40,581 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,581 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:40,581 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:40,582 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:40,582 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,582 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,582 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,582 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:40,582 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:40,582 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-12 19:10:40,583 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:40,583 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:40,583 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189070583 2023-07-12 19:10:40,585 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:40,585 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,587 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:40,587 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:40,587 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:40,587 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189100587 2023-07-12 19:10:40,587 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189160587 2023-07-12 19:10:40,588 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:10:40,588 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:40,590 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:40,590 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-12 19:10:40,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:40,591 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:40,838 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,838 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:40,838 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:40,838 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:40,840 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:40,841 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:40,841 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:40,841 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:40,841 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:40,843 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:40,845 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-12 19:10:40,846 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:10:40,846 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:10:40,847 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:40,848 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:40,848 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:40,849 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:40,849 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:40,850 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:40,850 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:40,850 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:40,850 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:40,850 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:40,851 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:40,851 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:40,851 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:40,852 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,852 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:40,852 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:40,852 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:40,852 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,852 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,852 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,852 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:40,852 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:40,852 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-07-12 19:10:40,853 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:40,854 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:40,854 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189070854 2023-07-12 19:10:40,855 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:40,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:40,858 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:40,859 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:40,859 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:40,859 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189100859 2023-07-12 19:10:40,859 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189160859 2023-07-12 19:10:40,859 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:10:40,859 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:40,862 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN] 2023-07-12 19:10:40,863 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:40,863 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-12 19:10:40,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:40,863 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:41,109 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,109 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:41,109 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:41,109 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:41,110 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:41,110 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:41,110 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:41,110 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:41,110 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:41,111 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 19:10:41,113 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-12 19:10:41,113 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:10:41,113 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:10:41,113 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-12 19:10:41,113 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:41,114 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:41,114 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:41,114 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:41,115 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:41,115 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:41,115 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:41,115 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:41,116 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:41,116 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:41,116 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:41,116 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:41,116 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:41,118 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,118 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:41,118 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:41,118 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:10:41,118 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:41,118 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,118 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,118 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,118 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:41,119 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:41,119 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:41,119 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-12 19:10:41,120 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:41,120 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:41,120 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6 to restore RIT 2023-07-12 19:10:41,121 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189071121 2023-07-12 19:10:41,122 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:41,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,124 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:10:41,125 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:41,126 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:41,126 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:41,126 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189101126 2023-07-12 19:10:41,126 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189161126 2023-07-12 19:10:41,126 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 19:10:41,126 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:41,132 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:41,132 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-12 19:10:41,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:41,132 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:41,376 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,377 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:41,377 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:41,377 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:41,377 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:41,377 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:41,377 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:41,377 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:41,378 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:41,378 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 19:10:41,380 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 19:10:41,380 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-12 19:10:41,380 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:41,380 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:41,381 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:41,381 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:41,381 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:41,381 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:41,382 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:41,382 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:41,382 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:41,383 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:41,383 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:41,383 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:41,384 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:41,385 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,385 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:10:41,385 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:41,385 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,385 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,385 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,386 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:41,386 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-12 19:10:41,387 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:41,388 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:41,388 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6 to restore RIT 2023-07-12 19:10:41,389 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189071389 2023-07-12 19:10:41,389 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:41,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,392 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:10:41,395 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:41,395 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:41,395 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:41,395 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189101395 2023-07-12 19:10:41,395 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189161395 2023-07-12 19:10:41,395 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 6 msec 2023-07-12 19:10:41,396 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:41,399 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:41,399 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-12 19:10:41,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:41,399 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:41,646 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,646 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:41,646 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:41,646 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:41,647 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:41,647 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:41,647 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:41,647 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:41,648 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:41,649 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:41,651 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-12 19:10:41,651 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-12 19:10:41,651 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:41,651 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:41,652 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:41,652 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:41,652 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:41,653 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:41,653 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:41,653 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:41,653 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:41,654 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:41,654 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:41,654 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:41,654 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:41,655 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,655 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:10:41,655 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:41,655 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,655 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,655 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,655 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:41,655 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 19:10:41,656 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:41,657 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:41,657 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6 to restore RIT 2023-07-12 19:10:41,657 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189071657 2023-07-12 19:10:41,658 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:41,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,661 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:10:41,663 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:41,663 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:41,663 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:41,663 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189101663 2023-07-12 19:10:41,664 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189161664 2023-07-12 19:10:41,664 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-07-12 19:10:41,664 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:41,666 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:41,667 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-12 19:10:41,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:41,667 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:41,914 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,914 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:41,914 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:41,914 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:41,915 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:41,915 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:41,915 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:41,915 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:41,915 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:41,916 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 19:10:41,916 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-12 19:10:41,916 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-12 19:10:41,916 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:41,917 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:41,917 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:41,917 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:41,917 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:41,918 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:41,918 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:41,918 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:41,918 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:41,918 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:41,918 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:41,918 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:41,919 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:41,919 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,919 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:10:41,919 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:41,919 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,919 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,919 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,920 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:10:41,920 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-12 19:10:41,920 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:41,921 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:41,921 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6 to restore RIT 2023-07-12 19:10:41,921 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189071921 2023-07-12 19:10:41,924 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:41,924 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:41,927 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:10:41,929 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:41,929 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:41,929 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:41,929 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189101929 2023-07-12 19:10:41,929 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189161929 2023-07-12 19:10:41,929 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-07-12 19:10:41,929 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:10:41,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e exec-time=4.8230 sec 2023-07-12 19:10:41,933 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:10:41,933 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN; state=CLOSED, location=jenkins-hbase20.apache.org,33255,1689189020635; forceNewPlan=false, retain=true 2023-07-12 19:10:42,084 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 19:10:42,086 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=eb3ad527ad66eb4244896aace6f290b6, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:42,086 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189042086"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189042086"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189042086"}]},"ts":"1689189042086"} 2023-07-12 19:10:42,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:10:42,090 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:10:42,090 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:10:42,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:10:42,090 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:10:42,179 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:10:42,180 WARN [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-07-12 19:10:42,180 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:10:42,180 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:10:42,180 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:10:42,182 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:10:42,182 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:10:42,183 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:10:42,183 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:10:42,183 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:10:42,184 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:10:42,187 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-07-12 19:10:42,187 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:10:42,188 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:10:42,188 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:10:42,189 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:10:42,189 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:42,190 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:10:42,190 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:42,190 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:10:42,190 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:42,191 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:10:42,191 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:42,191 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:10:42,191 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:42,191 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:10:42,191 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:10:42,191 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:10:42,191 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:10:42,192 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 19:10:42,192 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:10:42,193 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:10:42,193 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6 to restore RIT 2023-07-12 19:10:42,193 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189072193 2023-07-12 19:10:42,193 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:10:42,194 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:10:42,195 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:10:42,196 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=eb3ad527ad66eb4244896aace6f290b6, regionState=OPENING, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:10:42,196 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:10:42,196 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189102196 2023-07-12 19:10:42,196 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189162196 2023-07-12 19:10:42,196 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 19:10:42,196 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=eb3ad527ad66eb4244896aace6f290b6, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:42,196 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189042196"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189042196"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189042196"}]},"ts":"1689189042196"} 2023-07-12 19:10:42,198 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:10:42,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:42,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => eb3ad527ad66eb4244896aace6f290b6, NAME => 'testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.', STARTKEY => '', ENDKEY => ''} 2023-07-12 19:10:42,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:42,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:42,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:42,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:42,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:42,358 INFO [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:42,360 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1 2023-07-12 19:10:42,360 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1 2023-07-12 19:10:42,360 INFO [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eb3ad527ad66eb4244896aace6f290b6 columnFamilyName cf1 2023-07-12 19:10:42,373 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/203c1d7924984b6f86515610cab31aaa 2023-07-12 19:10:42,379 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/8bfbc793cd9845c29206d5bfdf50ca7a 2023-07-12 19:10:42,383 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9aee5e79a11b4886b6851b1477a76e2e 2023-07-12 19:10:42,388 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9d0b63720acc4af395b05a9a5c647ab8 2023-07-12 19:10:42,392 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/c39444aa379d411cabe6ffdd436460b5 2023-07-12 19:10:42,397 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/cf0b5ff74e784f6884f29717809791bd 2023-07-12 19:10:42,401 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/df375e2ea3414112949933e0590aacb1 2023-07-12 19:10:42,401 INFO [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(310): Store=eb3ad527ad66eb4244896aace6f290b6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:42,402 INFO [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:42,403 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2 2023-07-12 19:10:42,403 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2 2023-07-12 19:10:42,403 INFO [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eb3ad527ad66eb4244896aace6f290b6 columnFamilyName cf2 2023-07-12 19:10:42,410 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/57838bef137f4697a653f6dcdb94173a 2023-07-12 19:10:42,414 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/5c983e2c37cd4ab99fef58551f58b66e 2023-07-12 19:10:42,418 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/8aa2855ba2bf41818142629c6abeff33 2023-07-12 19:10:42,422 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/aaac648f7ddd41fab2a4c1523eee9bab 2023-07-12 19:10:42,427 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/c91903d0557f49259e4dc3e82d24e7cd 2023-07-12 19:10:42,430 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/e9b8e07648354a4c93beed7b7f2933c6 2023-07-12 19:10:42,433 DEBUG [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/f86bd59364af4013a8141384d60fa08b 2023-07-12 19:10:42,433 INFO [StoreOpener-eb3ad527ad66eb4244896aace6f290b6-1] regionserver.HStore(310): Store=eb3ad527ad66eb4244896aace6f290b6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:42,434 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:42,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:42,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.splits 2023-07-12 19:10:42,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:10:42,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:42,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened eb3ad527ad66eb4244896aace6f290b6; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65701501, jitterRate=-0.02097134292125702}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:10:42,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for eb3ad527ad66eb4244896aace6f290b6: 2023-07-12 19:10:42,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6., pid=48, masterSystemTime=1689189042350 2023-07-12 19:10:42,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:42,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:10:42,441 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:10:42,441 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:10:42,442 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-12 19:10:42,442 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-12 19:10:42,442 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HStore(1912): eb3ad527ad66eb4244896aace6f290b6/cf1 is initiating minor compaction (all files) 2023-07-12 19:10:42,442 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HStore(1912): eb3ad527ad66eb4244896aace6f290b6/cf2 is initiating minor compaction (all files) 2023-07-12 19:10:42,442 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of eb3ad527ad66eb4244896aace6f290b6/cf1 in testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:42,443 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HRegion(2259): Starting compaction of eb3ad527ad66eb4244896aace6f290b6/cf2 in testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:42,443 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/203c1d7924984b6f86515610cab31aaa, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/c39444aa379d411cabe6ffdd436460b5, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9d0b63720acc4af395b05a9a5c647ab8, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9aee5e79a11b4886b6851b1477a76e2e, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/cf0b5ff74e784f6884f29717809791bd, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/df375e2ea3414112949933e0590aacb1, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/8bfbc793cd9845c29206d5bfdf50ca7a] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp, totalSize=34.9 K 2023-07-12 19:10:42,443 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/5c983e2c37cd4ab99fef58551f58b66e, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/c91903d0557f49259e4dc3e82d24e7cd, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/aaac648f7ddd41fab2a4c1523eee9bab, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/57838bef137f4697a653f6dcdb94173a, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/e9b8e07648354a4c93beed7b7f2933c6, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/8aa2855ba2bf41818142629c6abeff33, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/f86bd59364af4013a8141384d60fa08b] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp, totalSize=34.9 K 2023-07-12 19:10:42,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:42,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:42,444 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting 203c1d7924984b6f86515610cab31aaa, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689189035650 2023-07-12 19:10:42,444 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting 5c983e2c37cd4ab99fef58551f58b66e, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689189035650 2023-07-12 19:10:42,444 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=eb3ad527ad66eb4244896aace6f290b6, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:42,444 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189042444"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189042444"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189042444"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189042444"}]},"ts":"1689189042444"} 2023-07-12 19:10:42,444 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting c39444aa379d411cabe6ffdd436460b5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689189035833 2023-07-12 19:10:42,444 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting c91903d0557f49259e4dc3e82d24e7cd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689189035833 2023-07-12 19:10:42,444 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting 9d0b63720acc4af395b05a9a5c647ab8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689189036014 2023-07-12 19:10:42,444 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting aaac648f7ddd41fab2a4c1523eee9bab, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689189036014 2023-07-12 19:10:42,445 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting 9aee5e79a11b4886b6851b1477a76e2e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689189036215 2023-07-12 19:10:42,445 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting 57838bef137f4697a653f6dcdb94173a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689189036215 2023-07-12 19:10:42,445 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting cf0b5ff74e784f6884f29717809791bd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689189036401 2023-07-12 19:10:42,445 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting e9b8e07648354a4c93beed7b7f2933c6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689189036401 2023-07-12 19:10:42,445 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting df375e2ea3414112949933e0590aacb1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689189036626 2023-07-12 19:10:42,446 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting 8aa2855ba2bf41818142629c6abeff33, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689189036626 2023-07-12 19:10:42,446 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting 8bfbc793cd9845c29206d5bfdf50ca7a, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689189036837 2023-07-12 19:10:42,446 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting f86bd59364af4013a8141384d60fa08b, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689189036837 2023-07-12 19:10:42,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-07-12 19:10:42,447 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-07-12 19:10:42,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 in 247 msec 2023-07-12 19:10:42,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN in 1.5870 sec 2023-07-12 19:10:42,450 DEBUG [Listener at localhost.localdomain/40405] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-07-12 19:10:42,452 INFO [Listener at localhost.localdomain/40405] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-07-12 19:10:42,455 INFO [Listener at localhost.localdomain/40405] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-07-12 19:10:42,455 INFO [Listener at localhost.localdomain/40405] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-07-12 19:10:42,456 INFO [Listener at localhost.localdomain/40405] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-07-12 19:10:42,471 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] throttle.PressureAwareThroughputController(145): eb3ad527ad66eb4244896aace6f290b6#cf2#compaction#50 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:42,480 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] throttle.PressureAwareThroughputController(145): eb3ad527ad66eb4244896aace6f290b6#cf1#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:42,492 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf2/affca5bbb8d44a2f86bf16509cd8d091 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/affca5bbb8d44a2f86bf16509cd8d091 2023-07-12 19:10:42,497 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in eb3ad527ad66eb4244896aace6f290b6/cf2 of eb3ad527ad66eb4244896aace6f290b6 into affca5bbb8d44a2f86bf16509cd8d091(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:42,497 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for eb3ad527ad66eb4244896aace6f290b6: 2023-07-12 19:10:42,497 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6., storeName=eb3ad527ad66eb4244896aace6f290b6/cf2, priority=9, startTime=1689189042440; duration=0sec 2023-07-12 19:10:42,502 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:42,510 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/.tmp/cf1/9414832ea7744291b9b433648b39605f as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9414832ea7744291b9b433648b39605f 2023-07-12 19:10:42,515 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in eb3ad527ad66eb4244896aace6f290b6/cf1 of eb3ad527ad66eb4244896aace6f290b6 into 9414832ea7744291b9b433648b39605f(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:42,516 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for eb3ad527ad66eb4244896aace6f290b6: 2023-07-12 19:10:42,516 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6., storeName=eb3ad527ad66eb4244896aace6f290b6/cf1, priority=9, startTime=1689189042440; duration=0sec 2023-07-12 19:10:42,516 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:42,520 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:10:42,522 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-07-12 19:10:42,522 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testRollbackAndDoubleExecution 2023-07-12 19:10:42,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:42,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-12 19:10:42,526 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189042526"}]},"ts":"1689189042526"} 2023-07-12 19:10:42,527 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-07-12 19:10:42,528 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-07-12 19:10:42,529 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN}] 2023-07-12 19:10:42,529 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN 2023-07-12 19:10:42,530 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:42,530 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189042530"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189042530"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189042530"}]},"ts":"1689189042530"} 2023-07-12 19:10:42,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:10:42,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-12 19:10:42,684 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:42,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing eb3ad527ad66eb4244896aace6f290b6, disabling compactions & flushes 2023-07-12 19:10:42,684 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:42,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:42,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. after waiting 0 ms 2023-07-12 19:10:42,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:42,686 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/203c1d7924984b6f86515610cab31aaa, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/c39444aa379d411cabe6ffdd436460b5, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9d0b63720acc4af395b05a9a5c647ab8, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9aee5e79a11b4886b6851b1477a76e2e, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/cf0b5ff74e784f6884f29717809791bd, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/df375e2ea3414112949933e0590aacb1, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/8bfbc793cd9845c29206d5bfdf50ca7a] to archive 2023-07-12 19:10:42,689 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:10:42,694 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/203c1d7924984b6f86515610cab31aaa to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/203c1d7924984b6f86515610cab31aaa 2023-07-12 19:10:42,697 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/c39444aa379d411cabe6ffdd436460b5 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/c39444aa379d411cabe6ffdd436460b5 2023-07-12 19:10:42,699 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9d0b63720acc4af395b05a9a5c647ab8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9d0b63720acc4af395b05a9a5c647ab8 2023-07-12 19:10:42,701 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9aee5e79a11b4886b6851b1477a76e2e to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9aee5e79a11b4886b6851b1477a76e2e 2023-07-12 19:10:42,703 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/cf0b5ff74e784f6884f29717809791bd to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/cf0b5ff74e784f6884f29717809791bd 2023-07-12 19:10:42,705 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/df375e2ea3414112949933e0590aacb1 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/df375e2ea3414112949933e0590aacb1 2023-07-12 19:10:42,707 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/8bfbc793cd9845c29206d5bfdf50ca7a to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/8bfbc793cd9845c29206d5bfdf50ca7a 2023-07-12 19:10:42,713 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/5c983e2c37cd4ab99fef58551f58b66e, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/c91903d0557f49259e4dc3e82d24e7cd, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/aaac648f7ddd41fab2a4c1523eee9bab, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/57838bef137f4697a653f6dcdb94173a, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/e9b8e07648354a4c93beed7b7f2933c6, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/8aa2855ba2bf41818142629c6abeff33, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/f86bd59364af4013a8141384d60fa08b] to archive 2023-07-12 19:10:42,714 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:10:42,716 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/5c983e2c37cd4ab99fef58551f58b66e to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/5c983e2c37cd4ab99fef58551f58b66e 2023-07-12 19:10:42,718 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/c91903d0557f49259e4dc3e82d24e7cd to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/c91903d0557f49259e4dc3e82d24e7cd 2023-07-12 19:10:42,719 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/aaac648f7ddd41fab2a4c1523eee9bab to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/aaac648f7ddd41fab2a4c1523eee9bab 2023-07-12 19:10:42,720 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/57838bef137f4697a653f6dcdb94173a to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/57838bef137f4697a653f6dcdb94173a 2023-07-12 19:10:42,721 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/e9b8e07648354a4c93beed7b7f2933c6 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/e9b8e07648354a4c93beed7b7f2933c6 2023-07-12 19:10:42,722 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/8aa2855ba2bf41818142629c6abeff33 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/8aa2855ba2bf41818142629c6abeff33 2023-07-12 19:10:42,723 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/f86bd59364af4013a8141384d60fa08b to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/f86bd59364af4013a8141384d60fa08b 2023-07-12 19:10:42,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 19:10:42,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:42,727 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6. 2023-07-12 19:10:42,728 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for eb3ad527ad66eb4244896aace6f290b6: 2023-07-12 19:10:42,729 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:42,730 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=eb3ad527ad66eb4244896aace6f290b6, regionState=CLOSED 2023-07-12 19:10:42,730 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189042729"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189042729"}]},"ts":"1689189042729"} 2023-07-12 19:10:42,732 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-07-12 19:10:42,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure eb3ad527ad66eb4244896aace6f290b6, server=jenkins-hbase20.apache.org,33255,1689189020635 in 200 msec 2023-07-12 19:10:42,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-07-12 19:10:42,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, UNASSIGN in 205 msec 2023-07-12 19:10:42,735 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189042735"}]},"ts":"1689189042735"} 2023-07-12 19:10:42,736 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-07-12 19:10:42,737 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-07-12 19:10:42,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 214 msec 2023-07-12 19:10:42,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-12 19:10:42,830 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-07-12 19:10:42,831 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testRollbackAndDoubleExecution 2023-07-12 19:10:42,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:42,834 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:42,834 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:42,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-07-12 19:10:42,838 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:42,840 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/recovered.edits] 2023-07-12 19:10:42,843 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9414832ea7744291b9b433648b39605f to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf1/9414832ea7744291b9b433648b39605f 2023-07-12 19:10:42,845 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/affca5bbb8d44a2f86bf16509cd8d091 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/cf2/affca5bbb8d44a2f86bf16509cd8d091 2023-07-12 19:10:42,848 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/recovered.edits/90.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6/recovered.edits/90.seqid 2023-07-12 19:10:42,849 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackAndDoubleExecution/eb3ad527ad66eb4244896aace6f290b6 2023-07-12 19:10:42,849 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-07-12 19:10:42,852 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:42,854 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-07-12 19:10:42,855 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-07-12 19:10:42,856 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:42,856 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-07-12 19:10:42,857 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189042857"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:42,858 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-12 19:10:42,858 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => eb3ad527ad66eb4244896aace6f290b6, NAME => 'testRollbackAndDoubleExecution,,1689189035135.eb3ad527ad66eb4244896aace6f290b6.', STARTKEY => '', ENDKEY => ''}] 2023-07-12 19:10:42,858 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-07-12 19:10:42,858 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689189042858"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:42,860 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-07-12 19:10:42,861 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:10:42,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 31 msec 2023-07-12 19:10:42,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-07-12 19:10:42,937 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-07-12 19:10:42,959 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=402 (was 398) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:35030 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:47514 [Waiting for operation #38] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:35032 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:51864 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:60912 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:45850 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:60830 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:35440 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:45916 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:45620 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:51858 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:45860 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase20:33255-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:34992 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:47520 [Waiting for operation #55] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:34976 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1802283332_17 at /127.0.0.1:60856 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=874 (was 862) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=187 (was 204), ProcessCount=170 (was 170), AvailableMemoryMB=5848 (was 5863) 2023-07-12 19:10:42,978 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=402, OpenFileDescriptor=874, MaxFileDescriptor=60000, SystemLoadAverage=187, ProcessCount=170, AvailableMemoryMB=5847 2023-07-12 19:10:42,978 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:10:42,981 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 19:10:42,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 19:10:42,984 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:10:42,986 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 19:10:42,987 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 19:10:42,989 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c 2023-07-12 19:10:42,990 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c empty. 2023-07-12 19:10:42,990 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c 2023-07-12 19:10:42,990 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-07-12 19:10:43,004 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-07-12 19:10:43,006 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7675): creating {ENCODED => e841382c5b6af7d03ac7388d8251f64c, NAME => 'testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp 2023-07-12 19:10:43,021 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:43,021 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1604): Closing e841382c5b6af7d03ac7388d8251f64c, disabling compactions & flushes 2023-07-12 19:10:43,021 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c. 2023-07-12 19:10:43,021 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c. 2023-07-12 19:10:43,021 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c. after waiting 0 ms 2023-07-12 19:10:43,021 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c. 2023-07-12 19:10:43,021 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c. 2023-07-12 19:10:43,021 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1558): Region close journal for e841382c5b6af7d03ac7388d8251f64c: 2023-07-12 19:10:43,023 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 19:10:43,024 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1689189043024"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189043024"}]},"ts":"1689189043024"} 2023-07-12 19:10:43,025 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 19:10:43,026 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 19:10:43,026 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189043026"}]},"ts":"1689189043026"} 2023-07-12 19:10:43,027 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-07-12 19:10:43,029 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 19:10:43,030 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 19:10:43,030 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 19:10:43,030 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 19:10:43,030 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 19:10:43,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e841382c5b6af7d03ac7388d8251f64c, ASSIGN}] 2023-07-12 19:10:43,032 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e841382c5b6af7d03ac7388d8251f64c, ASSIGN 2023-07-12 19:10:43,032 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e841382c5b6af7d03ac7388d8251f64c, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,38353,1689189020667; forceNewPlan=false, retain=false 2023-07-12 19:10:43,183 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 19:10:43,185 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=e841382c5b6af7d03ac7388d8251f64c, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:43,185 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1689189043185"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189043185"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189043185"}]},"ts":"1689189043185"} 2023-07-12 19:10:43,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure e841382c5b6af7d03ac7388d8251f64c, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:43,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c. 2023-07-12 19:10:43,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e841382c5b6af7d03ac7388d8251f64c, NAME => 'testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c.', STARTKEY => '', ENDKEY => ''} 2023-07-12 19:10:43,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:43,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile e841382c5b6af7d03ac7388d8251f64c 2023-07-12 19:10:43,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:43,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for e841382c5b6af7d03ac7388d8251f64c 2023-07-12 19:10:43,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for e841382c5b6af7d03ac7388d8251f64c 2023-07-12 19:10:43,356 INFO [StoreOpener-e841382c5b6af7d03ac7388d8251f64c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e841382c5b6af7d03ac7388d8251f64c 2023-07-12 19:10:43,358 DEBUG [StoreOpener-e841382c5b6af7d03ac7388d8251f64c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c/cf1 2023-07-12 19:10:43,358 DEBUG [StoreOpener-e841382c5b6af7d03ac7388d8251f64c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c/cf1 2023-07-12 19:10:43,358 INFO [StoreOpener-e841382c5b6af7d03ac7388d8251f64c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e841382c5b6af7d03ac7388d8251f64c columnFamilyName cf1 2023-07-12 19:10:43,359 INFO [StoreOpener-e841382c5b6af7d03ac7388d8251f64c-1] regionserver.HStore(310): Store=e841382c5b6af7d03ac7388d8251f64c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:43,359 INFO [StoreOpener-e841382c5b6af7d03ac7388d8251f64c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e841382c5b6af7d03ac7388d8251f64c 2023-07-12 19:10:43,361 DEBUG [StoreOpener-e841382c5b6af7d03ac7388d8251f64c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c/cf2 2023-07-12 19:10:43,361 DEBUG [StoreOpener-e841382c5b6af7d03ac7388d8251f64c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c/cf2 2023-07-12 19:10:43,361 INFO [StoreOpener-e841382c5b6af7d03ac7388d8251f64c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e841382c5b6af7d03ac7388d8251f64c columnFamilyName cf2 2023-07-12 19:10:43,362 INFO [StoreOpener-e841382c5b6af7d03ac7388d8251f64c-1] regionserver.HStore(310): Store=e841382c5b6af7d03ac7388d8251f64c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:43,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c 2023-07-12 19:10:43,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c 2023-07-12 19:10:43,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:10:43,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for e841382c5b6af7d03ac7388d8251f64c 2023-07-12 19:10:43,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 19:10:43,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened e841382c5b6af7d03ac7388d8251f64c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70020963, jitterRate=0.043393656611442566}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:10:43,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for e841382c5b6af7d03ac7388d8251f64c: 2023-07-12 19:10:43,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c., pid=55, masterSystemTime=1689189043344 2023-07-12 19:10:43,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c. 2023-07-12 19:10:43,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c. 2023-07-12 19:10:43,370 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=e841382c5b6af7d03ac7388d8251f64c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:43,370 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1689189043370"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189043370"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189043370"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189043370"}]},"ts":"1689189043370"} 2023-07-12 19:10:43,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-07-12 19:10:43,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure e841382c5b6af7d03ac7388d8251f64c, server=jenkins-hbase20.apache.org,38353,1689189020667 in 183 msec 2023-07-12 19:10:43,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-07-12 19:10:43,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e841382c5b6af7d03ac7388d8251f64c, ASSIGN in 343 msec 2023-07-12 19:10:43,375 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 19:10:43,375 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189043375"}]},"ts":"1689189043375"} 2023-07-12 19:10:43,377 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-07-12 19:10:43,378 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 19:10:43,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 396 msec 2023-07-12 19:10:43,485 DEBUG [Listener at localhost.localdomain/40405] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:43,487 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:43,489 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:43,489 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:43,489 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:43,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e841382c5b6af7d03ac7388d8251f64c, UNASSIGN}] 2023-07-12 19:10:43,494 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e841382c5b6af7d03ac7388d8251f64c, UNASSIGN 2023-07-12 19:10:43,495 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=e841382c5b6af7d03ac7388d8251f64c, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:43,495 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1689189043495"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189043495"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189043495"}]},"ts":"1689189043495"} 2023-07-12 19:10:43,497 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure e841382c5b6af7d03ac7388d8251f64c, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:43,649 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close e841382c5b6af7d03ac7388d8251f64c 2023-07-12 19:10:43,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing e841382c5b6af7d03ac7388d8251f64c, disabling compactions & flushes 2023-07-12 19:10:43,651 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c. 2023-07-12 19:10:43,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c. 2023-07-12 19:10:43,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c. after waiting 0 ms 2023-07-12 19:10:43,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c. 2023-07-12 19:10:43,658 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-12 19:10:43,659 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:43,659 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c. 2023-07-12 19:10:43,660 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for e841382c5b6af7d03ac7388d8251f64c: 2023-07-12 19:10:43,662 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed e841382c5b6af7d03ac7388d8251f64c 2023-07-12 19:10:43,663 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=e841382c5b6af7d03ac7388d8251f64c, regionState=CLOSED 2023-07-12 19:10:43,663 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1689189043663"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189043663"}]},"ts":"1689189043663"} 2023-07-12 19:10:43,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-07-12 19:10:43,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure e841382c5b6af7d03ac7388d8251f64c, server=jenkins-hbase20.apache.org,38353,1689189020667 in 167 msec 2023-07-12 19:10:43,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-07-12 19:10:43,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=e841382c5b6af7d03ac7388d8251f64c, UNASSIGN in 174 msec 2023-07-12 19:10:43,700 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/54b1b528e1fad095bb10622ca6766109/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-07-12 19:10:43,702 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/d6bc5978814690a86824058ca1615eeb/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-07-12 19:10:43,703 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1689189043703"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1689189043703"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1689189043703"}]},"ts":"1689189043703"} 2023-07-12 19:10:43,703 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189043703"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189043703"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189043703"}]},"ts":"1689189043703"} 2023-07-12 19:10:43,703 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189043703"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189043703"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189043703"}]},"ts":"1689189043703"} 2023-07-12 19:10:43,707 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=54b1b528e1fad095bb10622ca6766109, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d6bc5978814690a86824058ca1615eeb, ASSIGN}] 2023-07-12 19:10:43,708 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d6bc5978814690a86824058ca1615eeb, ASSIGN 2023-07-12 19:10:43,708 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d6bc5978814690a86824058ca1615eeb, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,38353,1689189020667; forceNewPlan=false, retain=false 2023-07-12 19:10:43,709 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=54b1b528e1fad095bb10622ca6766109, ASSIGN 2023-07-12 19:10:43,709 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=54b1b528e1fad095bb10622ca6766109, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,38353,1689189020667; forceNewPlan=false, retain=false 2023-07-12 19:10:43,858 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-12 19:10:43,862 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=d6bc5978814690a86824058ca1615eeb, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:43,862 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189043862"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189043862"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189043862"}]},"ts":"1689189043862"} 2023-07-12 19:10:43,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE; OpenRegionProcedure d6bc5978814690a86824058ca1615eeb, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:43,871 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=54b1b528e1fad095bb10622ca6766109, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:43,871 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189043871"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189043871"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189043871"}]},"ts":"1689189043871"} 2023-07-12 19:10:43,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=59, state=RUNNABLE; OpenRegionProcedure 54b1b528e1fad095bb10622ca6766109, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:44,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109. 2023-07-12 19:10:44,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 54b1b528e1fad095bb10622ca6766109, NAME => 'testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109.', STARTKEY => '', ENDKEY => '41'} 2023-07-12 19:10:44,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:44,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 54b1b528e1fad095bb10622ca6766109 2023-07-12 19:10:44,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:44,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 54b1b528e1fad095bb10622ca6766109 2023-07-12 19:10:44,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 54b1b528e1fad095bb10622ca6766109 2023-07-12 19:10:44,030 INFO [StoreOpener-54b1b528e1fad095bb10622ca6766109-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 54b1b528e1fad095bb10622ca6766109 2023-07-12 19:10:44,032 DEBUG [StoreOpener-54b1b528e1fad095bb10622ca6766109-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/54b1b528e1fad095bb10622ca6766109/cf1 2023-07-12 19:10:44,032 DEBUG [StoreOpener-54b1b528e1fad095bb10622ca6766109-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/54b1b528e1fad095bb10622ca6766109/cf1 2023-07-12 19:10:44,033 INFO [StoreOpener-54b1b528e1fad095bb10622ca6766109-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 54b1b528e1fad095bb10622ca6766109 columnFamilyName cf1 2023-07-12 19:10:44,034 INFO [StoreOpener-54b1b528e1fad095bb10622ca6766109-1] regionserver.HStore(310): Store=54b1b528e1fad095bb10622ca6766109/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:44,034 INFO [StoreOpener-54b1b528e1fad095bb10622ca6766109-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 54b1b528e1fad095bb10622ca6766109 2023-07-12 19:10:44,036 DEBUG [StoreOpener-54b1b528e1fad095bb10622ca6766109-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/54b1b528e1fad095bb10622ca6766109/cf2 2023-07-12 19:10:44,036 DEBUG [StoreOpener-54b1b528e1fad095bb10622ca6766109-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/54b1b528e1fad095bb10622ca6766109/cf2 2023-07-12 19:10:44,036 INFO [StoreOpener-54b1b528e1fad095bb10622ca6766109-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 54b1b528e1fad095bb10622ca6766109 columnFamilyName cf2 2023-07-12 19:10:44,037 INFO [StoreOpener-54b1b528e1fad095bb10622ca6766109-1] regionserver.HStore(310): Store=54b1b528e1fad095bb10622ca6766109/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:44,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/54b1b528e1fad095bb10622ca6766109 2023-07-12 19:10:44,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/54b1b528e1fad095bb10622ca6766109 2023-07-12 19:10:44,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:10:44,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 54b1b528e1fad095bb10622ca6766109 2023-07-12 19:10:44,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 54b1b528e1fad095bb10622ca6766109; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60644538, jitterRate=-0.09632596373558044}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:10:44,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 54b1b528e1fad095bb10622ca6766109: 2023-07-12 19:10:44,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109., pid=62, masterSystemTime=1689189044019 2023-07-12 19:10:44,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109. 2023-07-12 19:10:44,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109. 2023-07-12 19:10:44,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb. 2023-07-12 19:10:44,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d6bc5978814690a86824058ca1615eeb, NAME => 'testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb.', STARTKEY => '41', ENDKEY => ''} 2023-07-12 19:10:44,050 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=54b1b528e1fad095bb10622ca6766109, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:44,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:44,050 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189044050"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189044050"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189044050"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189044050"}]},"ts":"1689189044050"} 2023-07-12 19:10:44,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:44,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:44,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:44,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:44,052 INFO [StoreOpener-d6bc5978814690a86824058ca1615eeb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:44,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=59 2023-07-12 19:10:44,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=59, state=SUCCESS; OpenRegionProcedure 54b1b528e1fad095bb10622ca6766109, server=jenkins-hbase20.apache.org,38353,1689189020667 in 180 msec 2023-07-12 19:10:44,055 DEBUG [StoreOpener-d6bc5978814690a86824058ca1615eeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/d6bc5978814690a86824058ca1615eeb/cf1 2023-07-12 19:10:44,055 DEBUG [StoreOpener-d6bc5978814690a86824058ca1615eeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/d6bc5978814690a86824058ca1615eeb/cf1 2023-07-12 19:10:44,055 INFO [StoreOpener-d6bc5978814690a86824058ca1615eeb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d6bc5978814690a86824058ca1615eeb columnFamilyName cf1 2023-07-12 19:10:44,056 INFO [StoreOpener-d6bc5978814690a86824058ca1615eeb-1] regionserver.HStore(310): Store=d6bc5978814690a86824058ca1615eeb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:44,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=54b1b528e1fad095bb10622ca6766109, ASSIGN in 348 msec 2023-07-12 19:10:44,056 INFO [StoreOpener-d6bc5978814690a86824058ca1615eeb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:44,058 DEBUG [StoreOpener-d6bc5978814690a86824058ca1615eeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/d6bc5978814690a86824058ca1615eeb/cf2 2023-07-12 19:10:44,058 DEBUG [StoreOpener-d6bc5978814690a86824058ca1615eeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/d6bc5978814690a86824058ca1615eeb/cf2 2023-07-12 19:10:44,058 INFO [StoreOpener-d6bc5978814690a86824058ca1615eeb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d6bc5978814690a86824058ca1615eeb columnFamilyName cf2 2023-07-12 19:10:44,059 INFO [StoreOpener-d6bc5978814690a86824058ca1615eeb-1] regionserver.HStore(310): Store=d6bc5978814690a86824058ca1615eeb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:44,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:44,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:44,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:10:44,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:44,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened d6bc5978814690a86824058ca1615eeb; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67015046, jitterRate=-0.0013979971408843994}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:10:44,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for d6bc5978814690a86824058ca1615eeb: 2023-07-12 19:10:44,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb., pid=61, masterSystemTime=1689189044019 2023-07-12 19:10:44,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb. 2023-07-12 19:10:44,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb. 2023-07-12 19:10:44,068 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=d6bc5978814690a86824058ca1615eeb, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:44,068 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189044068"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189044068"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189044068"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189044068"}]},"ts":"1689189044068"} 2023-07-12 19:10:44,071 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-07-12 19:10:44,071 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; OpenRegionProcedure d6bc5978814690a86824058ca1615eeb, server=jenkins-hbase20.apache.org,38353,1689189020667 in 204 msec 2023-07-12 19:10:44,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=56 2023-07-12 19:10:44,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d6bc5978814690a86824058ca1615eeb, ASSIGN in 364 msec 2023-07-12 19:10:44,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb in 587 msec 2023-07-12 19:10:44,248 DEBUG [Listener at localhost.localdomain/40405] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-12 19:10:44,251 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:10:44,253 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-07-12 19:10:44,254 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitTableRegionNoStoreFile 2023-07-12 19:10:44,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:10:44,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-07-12 19:10:44,258 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189044258"}]},"ts":"1689189044258"} 2023-07-12 19:10:44,259 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-07-12 19:10:44,261 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-07-12 19:10:44,262 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=54b1b528e1fad095bb10622ca6766109, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d6bc5978814690a86824058ca1615eeb, UNASSIGN}] 2023-07-12 19:10:44,263 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d6bc5978814690a86824058ca1615eeb, UNASSIGN 2023-07-12 19:10:44,263 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=d6bc5978814690a86824058ca1615eeb, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:44,264 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189044263"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189044263"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189044263"}]},"ts":"1689189044263"} 2023-07-12 19:10:44,265 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure d6bc5978814690a86824058ca1615eeb, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:44,268 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=54b1b528e1fad095bb10622ca6766109, UNASSIGN 2023-07-12 19:10:44,270 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=54b1b528e1fad095bb10622ca6766109, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:44,270 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189044270"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189044270"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189044270"}]},"ts":"1689189044270"} 2023-07-12 19:10:44,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 54b1b528e1fad095bb10622ca6766109, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:44,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-07-12 19:10:44,419 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 54b1b528e1fad095bb10622ca6766109 2023-07-12 19:10:44,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 54b1b528e1fad095bb10622ca6766109, disabling compactions & flushes 2023-07-12 19:10:44,421 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109. 2023-07-12 19:10:44,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109. 2023-07-12 19:10:44,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109. after waiting 0 ms 2023-07-12 19:10:44,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109. 2023-07-12 19:10:44,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/54b1b528e1fad095bb10622ca6766109/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-07-12 19:10:44,431 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:44,431 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109. 2023-07-12 19:10:44,431 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 54b1b528e1fad095bb10622ca6766109: 2023-07-12 19:10:44,433 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 54b1b528e1fad095bb10622ca6766109 2023-07-12 19:10:44,433 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:44,434 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing d6bc5978814690a86824058ca1615eeb, disabling compactions & flushes 2023-07-12 19:10:44,434 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb. 2023-07-12 19:10:44,434 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb. 2023-07-12 19:10:44,435 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb. after waiting 0 ms 2023-07-12 19:10:44,435 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb. 2023-07-12 19:10:44,435 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=54b1b528e1fad095bb10622ca6766109, regionState=CLOSED 2023-07-12 19:10:44,435 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189044435"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189044435"}]},"ts":"1689189044435"} 2023-07-12 19:10:44,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-07-12 19:10:44,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 54b1b528e1fad095bb10622ca6766109, server=jenkins-hbase20.apache.org,38353,1689189020667 in 164 msec 2023-07-12 19:10:44,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionNoStoreFile/d6bc5978814690a86824058ca1615eeb/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-07-12 19:10:44,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:44,439 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb. 2023-07-12 19:10:44,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for d6bc5978814690a86824058ca1615eeb: 2023-07-12 19:10:44,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=54b1b528e1fad095bb10622ca6766109, UNASSIGN in 176 msec 2023-07-12 19:10:44,440 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:44,440 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=d6bc5978814690a86824058ca1615eeb, regionState=CLOSED 2023-07-12 19:10:44,440 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189044440"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189044440"}]},"ts":"1689189044440"} 2023-07-12 19:10:44,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-07-12 19:10:44,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure d6bc5978814690a86824058ca1615eeb, server=jenkins-hbase20.apache.org,38353,1689189020667 in 177 msec 2023-07-12 19:10:44,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-07-12 19:10:44,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d6bc5978814690a86824058ca1615eeb, UNASSIGN in 181 msec 2023-07-12 19:10:44,445 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189044445"}]},"ts":"1689189044445"} 2023-07-12 19:10:44,447 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-07-12 19:10:44,448 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-07-12 19:10:44,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 194 msec 2023-07-12 19:10:44,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-07-12 19:10:44,561 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-07-12 19:10:44,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitTableRegionNoStoreFile 2023-07-12 19:10:44,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:10:44,568 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:10:44,570 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:10:44,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-07-12 19:10:44,579 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c 2023-07-12 19:10:44,579 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:44,579 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/54b1b528e1fad095bb10622ca6766109 2023-07-12 19:10:44,580 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/54b1b528e1fad095bb10622ca6766109/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/54b1b528e1fad095bb10622ca6766109/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/54b1b528e1fad095bb10622ca6766109/recovered.edits] 2023-07-12 19:10:44,580 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/d6bc5978814690a86824058ca1615eeb/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/d6bc5978814690a86824058ca1615eeb/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/d6bc5978814690a86824058ca1615eeb/recovered.edits] 2023-07-12 19:10:44,581 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c/recovered.edits] 2023-07-12 19:10:44,586 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/54b1b528e1fad095bb10622ca6766109/recovered.edits/7.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionNoStoreFile/54b1b528e1fad095bb10622ca6766109/recovered.edits/7.seqid 2023-07-12 19:10:44,586 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/d6bc5978814690a86824058ca1615eeb/recovered.edits/7.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionNoStoreFile/d6bc5978814690a86824058ca1615eeb/recovered.edits/7.seqid 2023-07-12 19:10:44,586 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c/recovered.edits/4.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c/recovered.edits/4.seqid 2023-07-12 19:10:44,586 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/54b1b528e1fad095bb10622ca6766109 2023-07-12 19:10:44,586 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/d6bc5978814690a86824058ca1615eeb 2023-07-12 19:10:44,586 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionNoStoreFile/e841382c5b6af7d03ac7388d8251f64c 2023-07-12 19:10:44,586 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-07-12 19:10:44,589 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:10:44,590 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-07-12 19:10:44,592 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-07-12 19:10:44,593 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:10:44,593 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-07-12 19:10:44,593 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189044593"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:44,593 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189044593"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:44,593 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189044593"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:44,595 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-12 19:10:44,595 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e841382c5b6af7d03ac7388d8251f64c, NAME => 'testSplitTableRegionNoStoreFile,,1689189042982.e841382c5b6af7d03ac7388d8251f64c.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 54b1b528e1fad095bb10622ca6766109, NAME => 'testSplitTableRegionNoStoreFile,,1689189043486.54b1b528e1fad095bb10622ca6766109.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => d6bc5978814690a86824058ca1615eeb, NAME => 'testSplitTableRegionNoStoreFile,41,1689189043486.d6bc5978814690a86824058ca1615eeb.', STARTKEY => '41', ENDKEY => ''}] 2023-07-12 19:10:44,595 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-07-12 19:10:44,595 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689189044595"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:44,597 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-07-12 19:10:44,598 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:10:44,599 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 35 msec 2023-07-12 19:10:44,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-07-12 19:10:44,672 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-07-12 19:10:44,689 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=406 (was 402) Potentially hanging thread: hconnection-0x49452fc5-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-170374772_17 at /127.0.0.1:45620 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-170374772_17 at /127.0.0.1:34992 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=874 (was 874), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=188 (was 187) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=5847 (was 5847) 2023-07-12 19:10:44,706 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=406, OpenFileDescriptor=874, MaxFileDescriptor=60000, SystemLoadAverage=188, ProcessCount=170, AvailableMemoryMB=5846 2023-07-12 19:10:44,707 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:10:44,709 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 19:10:44,710 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 19:10:44,711 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:44,713 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 19:10:44,713 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 19:10:44,715 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:44,716 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0 empty. 2023-07-12 19:10:44,716 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:44,716 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-07-12 19:10:44,729 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-07-12 19:10:44,731 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 0e61b0eec6c8d265f5fb01657ce1d2a0, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp 2023-07-12 19:10:44,743 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:44,743 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1604): Closing 0e61b0eec6c8d265f5fb01657ce1d2a0, disabling compactions & flushes 2023-07-12 19:10:44,743 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:44,743 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:44,743 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. after waiting 0 ms 2023-07-12 19:10:44,743 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:44,743 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:44,743 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 0e61b0eec6c8d265f5fb01657ce1d2a0: 2023-07-12 19:10:44,745 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 19:10:44,746 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1689189044746"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189044746"}]},"ts":"1689189044746"} 2023-07-12 19:10:44,747 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 19:10:44,748 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 19:10:44,748 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189044748"}]},"ts":"1689189044748"} 2023-07-12 19:10:44,749 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-07-12 19:10:44,751 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 19:10:44,752 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 19:10:44,752 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 19:10:44,752 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 19:10:44,752 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 19:10:44,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0e61b0eec6c8d265f5fb01657ce1d2a0, ASSIGN}] 2023-07-12 19:10:44,753 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0e61b0eec6c8d265f5fb01657ce1d2a0, ASSIGN 2023-07-12 19:10:44,754 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0e61b0eec6c8d265f5fb01657ce1d2a0, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,38353,1689189020667; forceNewPlan=false, retain=false 2023-07-12 19:10:44,904 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 19:10:44,906 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=0e61b0eec6c8d265f5fb01657ce1d2a0, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:44,906 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1689189044906"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189044906"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189044906"}]},"ts":"1689189044906"} 2023-07-12 19:10:44,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 0e61b0eec6c8d265f5fb01657ce1d2a0, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:45,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:45,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0e61b0eec6c8d265f5fb01657ce1d2a0, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0.', STARTKEY => '', ENDKEY => ''} 2023-07-12 19:10:45,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:45,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:45,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:45,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:45,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:45,069 INFO [StoreOpener-0e61b0eec6c8d265f5fb01657ce1d2a0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:45,071 DEBUG [StoreOpener-0e61b0eec6c8d265f5fb01657ce1d2a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1 2023-07-12 19:10:45,071 DEBUG [StoreOpener-0e61b0eec6c8d265f5fb01657ce1d2a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1 2023-07-12 19:10:45,072 INFO [StoreOpener-0e61b0eec6c8d265f5fb01657ce1d2a0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0e61b0eec6c8d265f5fb01657ce1d2a0 columnFamilyName cf1 2023-07-12 19:10:45,073 INFO [StoreOpener-0e61b0eec6c8d265f5fb01657ce1d2a0-1] regionserver.HStore(310): Store=0e61b0eec6c8d265f5fb01657ce1d2a0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:45,073 INFO [StoreOpener-0e61b0eec6c8d265f5fb01657ce1d2a0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:45,075 DEBUG [StoreOpener-0e61b0eec6c8d265f5fb01657ce1d2a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2 2023-07-12 19:10:45,075 DEBUG [StoreOpener-0e61b0eec6c8d265f5fb01657ce1d2a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2 2023-07-12 19:10:45,075 INFO [StoreOpener-0e61b0eec6c8d265f5fb01657ce1d2a0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0e61b0eec6c8d265f5fb01657ce1d2a0 columnFamilyName cf2 2023-07-12 19:10:45,076 INFO [StoreOpener-0e61b0eec6c8d265f5fb01657ce1d2a0-1] regionserver.HStore(310): Store=0e61b0eec6c8d265f5fb01657ce1d2a0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:45,077 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:45,077 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:45,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:10:45,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:45,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 19:10:45,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 0e61b0eec6c8d265f5fb01657ce1d2a0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74771411, jitterRate=0.11418084800243378}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:10:45,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 0e61b0eec6c8d265f5fb01657ce1d2a0: 2023-07-12 19:10:45,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0., pid=71, masterSystemTime=1689189045064 2023-07-12 19:10:45,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:45,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:45,088 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=0e61b0eec6c8d265f5fb01657ce1d2a0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,088 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1689189045088"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189045088"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189045088"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189045088"}]},"ts":"1689189045088"} 2023-07-12 19:10:45,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-07-12 19:10:45,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 0e61b0eec6c8d265f5fb01657ce1d2a0, server=jenkins-hbase20.apache.org,38353,1689189020667 in 179 msec 2023-07-12 19:10:45,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-07-12 19:10:45,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0e61b0eec6c8d265f5fb01657ce1d2a0, ASSIGN in 339 msec 2023-07-12 19:10:45,093 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 19:10:45,093 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189045093"}]},"ts":"1689189045093"} 2023-07-12 19:10:45,094 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-07-12 19:10:45,096 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 19:10:45,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 386 msec 2023-07-12 19:10:45,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:45,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,225 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:45,225 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,225 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 19:10:45,225 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 19:10:45,226 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,226 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,227 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,227 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,227 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,227 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,227 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,227 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:45,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:45,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:45,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,228 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,228 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:45,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,228 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,228 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:45,228 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:45,228 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,228 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,228 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:45,228 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,228 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:45,228 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:45,229 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:45,228 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:45,228 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:45,229 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:45,229 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:45,229 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:45,229 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:45,229 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:45,229 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. started... 2023-07-12 19:10:45,230 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0e61b0eec6c8d265f5fb01657ce1d2a0 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 19:10:45,230 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,231 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,231 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,231 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:45,231 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,231 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:45,248 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/090cc5071bb041f69ad5258a25e1429d 2023-07-12 19:10:45,265 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/236c2e5d04814c849ee02ba7488bd884 2023-07-12 19:10:45,270 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/090cc5071bb041f69ad5258a25e1429d as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/090cc5071bb041f69ad5258a25e1429d 2023-07-12 19:10:45,275 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/090cc5071bb041f69ad5258a25e1429d, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:10:45,276 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/236c2e5d04814c849ee02ba7488bd884 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/236c2e5d04814c849ee02ba7488bd884 2023-07-12 19:10:45,281 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/236c2e5d04814c849ee02ba7488bd884, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:10:45,282 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 0e61b0eec6c8d265f5fb01657ce1d2a0 in 52ms, sequenceid=6, compaction requested=false 2023-07-12 19:10:45,282 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:45,283 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0e61b0eec6c8d265f5fb01657ce1d2a0: 2023-07-12 19:10:45,283 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:45,283 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:45,283 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:45,283 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,283 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:45,283 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:45,284 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,284 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:45,285 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,285 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:45,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 19:10:45,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3dec206b[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:45,286 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 19:10:45,287 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,287 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,287 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,287 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,287 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,287 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:45,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,287 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:45,288 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:45,288 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:45,288 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:45,288 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:45,288 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:45,288 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:45,288 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:45,288 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:45,289 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:45,289 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,289 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:45,289 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,289 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:45,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:45,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:45,289 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:45,289 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,289 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,289 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:45,289 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:45,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:45,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:45,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:45,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:45,292 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:10:45,292 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 19:10:45,292 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:45,292 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:45,292 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,292 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:45,293 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,293 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,293 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,293 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,293 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,293 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,293 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:45,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,294 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,294 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,295 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,296 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,296 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:45,296 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,303 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,303 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,303 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,303 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,303 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:45,303 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:45,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,303 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,303 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,303 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,303 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,303 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,304 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,304 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:45,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,304 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:45,304 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,304 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,304 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,304 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,304 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,304 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,304 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,305 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,404 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:45,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:45,419 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:45,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:45,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,420 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:45,421 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,421 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 19:10:45,421 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 19:10:45,421 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,421 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,422 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,422 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,422 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,422 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,422 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,422 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:45,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,423 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:45,423 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:45,423 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:45,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,423 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:45,424 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:45,424 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:45,424 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:45,424 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,424 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,424 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:45,424 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:45,424 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:45,424 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:45,424 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:45,424 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:45,424 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:45,424 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. started... 2023-07-12 19:10:45,424 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0e61b0eec6c8d265f5fb01657ce1d2a0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:45,424 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,425 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,425 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,425 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:45,425 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,425 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:45,435 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/72ccb17830a04baabfa21b59aad14e21 2023-07-12 19:10:45,447 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/94277c2ac28b4a4a9d2f8a60045fcc25 2023-07-12 19:10:45,452 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/72ccb17830a04baabfa21b59aad14e21 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/72ccb17830a04baabfa21b59aad14e21 2023-07-12 19:10:45,455 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/72ccb17830a04baabfa21b59aad14e21, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:10:45,456 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/94277c2ac28b4a4a9d2f8a60045fcc25 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/94277c2ac28b4a4a9d2f8a60045fcc25 2023-07-12 19:10:45,460 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/94277c2ac28b4a4a9d2f8a60045fcc25, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:10:45,460 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0e61b0eec6c8d265f5fb01657ce1d2a0 in 36ms, sequenceid=19, compaction requested=false 2023-07-12 19:10:45,460 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0e61b0eec6c8d265f5fb01657ce1d2a0: 2023-07-12 19:10:45,460 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:45,460 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:45,461 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:45,461 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,461 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:45,461 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:45,462 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:45,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:45,462 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:45,462 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,462 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:45,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:45,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:45,463 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 19:10:45,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b28e1a[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:45,464 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 19:10:45,464 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,464 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,464 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,464 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,464 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,465 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,465 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:45,465 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:45,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,465 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:45,465 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:45,465 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:45,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,465 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:45,465 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:45,465 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:45,465 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:45,466 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:45,466 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,466 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:45,466 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:45,466 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,466 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:45,466 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:45,466 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,466 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:45,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:45,466 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:45,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:45,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:45,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:45,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:45,470 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:10:45,470 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 19:10:45,470 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:45,470 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:45,470 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,470 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:45,471 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,471 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,471 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,471 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,471 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:45,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:45,471 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,472 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,471 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,472 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,472 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:45,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:45,474 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,475 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,475 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,478 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,478 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,478 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,478 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,478 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,478 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:45,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:45,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,478 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,478 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,478 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,478 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,479 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,478 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:45,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,478 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,479 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:45,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,479 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,479 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,480 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,480 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,480 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,480 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,579 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:45,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:45,599 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:45,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:45,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,601 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:45,601 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,601 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 19:10:45,601 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 19:10:45,602 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,602 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,603 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,603 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,603 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,603 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,603 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,603 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:45,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:45,604 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,604 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,604 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:45,604 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,605 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,605 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:45,605 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:45,605 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:45,605 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:45,605 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,605 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,605 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:45,605 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:45,605 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,605 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:45,605 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:45,605 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:45,605 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:45,605 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:45,605 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:45,605 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:45,605 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. started... 2023-07-12 19:10:45,605 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0e61b0eec6c8d265f5fb01657ce1d2a0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:45,606 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,607 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,607 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,607 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:45,607 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,607 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:45,613 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f 2023-07-12 19:10:45,627 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/1b6e38155cce49ec9d7b8478b011d7f4 2023-07-12 19:10:45,632 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f 2023-07-12 19:10:45,636 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:10:45,637 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/1b6e38155cce49ec9d7b8478b011d7f4 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/1b6e38155cce49ec9d7b8478b011d7f4 2023-07-12 19:10:45,641 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/1b6e38155cce49ec9d7b8478b011d7f4, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:10:45,642 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0e61b0eec6c8d265f5fb01657ce1d2a0 in 37ms, sequenceid=32, compaction requested=true 2023-07-12 19:10:45,642 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0e61b0eec6c8d265f5fb01657ce1d2a0: 2023-07-12 19:10:45,642 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:45,642 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:45,642 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:45,642 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,642 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:45,642 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:45,653 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,653 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:45,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:45,653 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,653 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:45,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:45,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:45,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:45,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 19:10:45,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5efdf79b[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:45,657 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 19:10:45,657 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,669 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,669 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,669 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,669 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:45,670 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:45,670 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:45,669 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,670 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:45,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,670 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:45,670 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:45,670 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:45,670 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:45,670 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:45,671 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:45,672 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:45,672 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,672 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:45,673 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:45,673 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:45,673 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,673 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:45,673 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,673 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,674 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:45,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:45,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:45,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:45,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:45,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:45,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:45,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:10:45,680 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 19:10:45,680 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:45,680 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:45,680 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,680 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:45,686 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,686 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,686 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,686 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,686 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,686 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:45,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:45,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,686 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,686 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:45,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,687 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,687 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:45,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:45,689 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,689 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,689 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,693 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,693 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,693 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,693 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,693 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,693 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,693 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,694 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,694 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:45,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:45,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,693 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,693 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,694 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:45,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,694 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,694 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:45,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,694 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,694 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,694 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,695 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,695 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,695 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,794 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:45,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:45,811 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:45,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:45,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,813 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:45,813 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,813 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 19:10:45,813 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 19:10:45,813 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,814 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,814 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,814 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,814 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,814 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,815 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,815 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:45,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:45,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,815 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,816 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,816 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:45,816 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,816 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:45,816 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,816 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:45,816 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:45,816 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,816 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:45,816 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:45,816 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:45,816 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,816 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:45,816 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:45,816 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,816 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:45,816 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:45,816 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:45,816 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:45,816 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:45,816 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:45,816 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:45,816 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:45,816 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. started... 2023-07-12 19:10:45,817 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0e61b0eec6c8d265f5fb01657ce1d2a0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:45,817 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,817 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,817 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,817 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:45,817 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,817 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:45,830 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/7a4f6744ef3248e6932fb626a8d2c353 2023-07-12 19:10:45,842 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/525cd0743cad400192a410acf27e8a5e 2023-07-12 19:10:45,846 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/7a4f6744ef3248e6932fb626a8d2c353 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/7a4f6744ef3248e6932fb626a8d2c353 2023-07-12 19:10:45,850 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/7a4f6744ef3248e6932fb626a8d2c353, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:10:45,851 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/525cd0743cad400192a410acf27e8a5e as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/525cd0743cad400192a410acf27e8a5e 2023-07-12 19:10:45,854 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/525cd0743cad400192a410acf27e8a5e, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:10:45,855 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0e61b0eec6c8d265f5fb01657ce1d2a0 in 38ms, sequenceid=45, compaction requested=true 2023-07-12 19:10:45,855 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0e61b0eec6c8d265f5fb01657ce1d2a0: 2023-07-12 19:10:45,855 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:45,855 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:45,855 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:45,855 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,855 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:45,855 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:45,857 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,857 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:45,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:45,857 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,857 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:45,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:45,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:45,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:45,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 19:10:45,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62883dc8[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:45,859 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 19:10:45,859 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,859 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,859 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,859 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,860 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:45,860 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:45,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,860 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:45,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,860 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:45,860 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,860 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:45,860 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:45,860 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:45,860 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:45,860 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:45,860 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:45,861 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:45,861 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,861 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:45,861 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:45,861 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,861 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:45,862 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,862 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:45,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:45,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:45,862 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,863 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:45,865 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:10:45,865 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 19:10:45,865 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:45,865 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:45,865 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,865 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:45,866 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,866 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,866 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:45,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,866 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,866 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,866 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:45,867 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,867 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,867 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,867 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,869 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,869 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:45,870 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,872 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,872 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,872 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,872 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,872 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:45,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,872 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,872 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,872 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,872 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:45,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:45,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,872 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:45,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,872 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:45,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:45,872 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,872 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:45,873 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,873 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:45,873 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,873 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:45,873 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:45,873 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,873 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:45,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:45,874 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:45,874 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:45,974 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:45,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:45,998 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:46,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:46,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,000 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:46,000 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,000 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 19:10:46,000 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 19:10:46,000 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,000 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,001 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,001 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,001 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,001 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,002 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,002 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:46,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,002 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,002 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:46,002 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:46,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,003 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,003 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:46,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,003 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,003 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:46,003 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:46,003 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,003 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:46,003 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:46,003 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,003 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,003 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:46,003 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,003 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,003 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:46,003 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:46,003 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:46,003 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:46,003 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:46,003 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,003 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:46,003 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:46,003 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:46,003 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:46,003 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:46,004 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:46,004 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. started... 2023-07-12 19:10:46,004 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0e61b0eec6c8d265f5fb01657ce1d2a0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:46,004 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,004 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,004 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:46,004 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,005 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,005 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:46,016 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/45695b5932ee4511a54c041a405619f0 2023-07-12 19:10:46,029 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/a0c5a22c1efb4dc99a241faa5a9b3080 2023-07-12 19:10:46,035 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/45695b5932ee4511a54c041a405619f0 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/45695b5932ee4511a54c041a405619f0 2023-07-12 19:10:46,039 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/45695b5932ee4511a54c041a405619f0, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:10:46,040 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/a0c5a22c1efb4dc99a241faa5a9b3080 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/a0c5a22c1efb4dc99a241faa5a9b3080 2023-07-12 19:10:46,045 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/a0c5a22c1efb4dc99a241faa5a9b3080, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:10:46,046 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0e61b0eec6c8d265f5fb01657ce1d2a0 in 42ms, sequenceid=58, compaction requested=true 2023-07-12 19:10:46,046 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0e61b0eec6c8d265f5fb01657ce1d2a0: 2023-07-12 19:10:46,046 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:46,046 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:46,046 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:46,046 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,046 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:46,046 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:46,047 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,048 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:46,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:46,048 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,048 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:46,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:46,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:46,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:46,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 19:10:46,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e3ba68a[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:46,050 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 19:10:46,050 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,051 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,051 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,051 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,051 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:46,051 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,051 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:46,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,051 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:46,051 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:46,051 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:46,051 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:46,051 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:46,051 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:46,051 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:46,051 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:46,052 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:46,052 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,052 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:46,052 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:46,052 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,052 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:46,052 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:46,052 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,052 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:46,052 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:46,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:46,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:46,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:46,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:46,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:46,056 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:10:46,056 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 19:10:46,056 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:46,056 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:46,056 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,056 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:46,057 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,057 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,057 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,057 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,057 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,057 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,057 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:46,058 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:46,058 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,059 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:46,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,060 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:46,060 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,061 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,069 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,069 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,069 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,069 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,070 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,070 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,069 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,069 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:46,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:46,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,070 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,070 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,070 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:46,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,070 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:46,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,070 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,071 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,071 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,071 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,071 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,071 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,071 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,071 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,171 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:46,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:46,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:46,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:46,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,186 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:46,186 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,186 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 19:10:46,186 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 19:10:46,186 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,186 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,187 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,187 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,187 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,187 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,187 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,187 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:46,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,188 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,188 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:46,188 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,188 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:46,188 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:46,188 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,188 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,188 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:46,188 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,188 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,189 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:46,188 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:46,189 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:46,189 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:46,189 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:46,189 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:46,189 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:46,189 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:46,189 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:46,189 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:46,189 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. started... 2023-07-12 19:10:46,189 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0e61b0eec6c8d265f5fb01657ce1d2a0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:10:46,190 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,190 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,190 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,190 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:46,190 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,190 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:46,197 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/dc1d1e60c2b94a1291973f788a370e95 2023-07-12 19:10:46,208 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/d747715a45d847d59970893fa9599993 2023-07-12 19:10:46,212 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/dc1d1e60c2b94a1291973f788a370e95 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/dc1d1e60c2b94a1291973f788a370e95 2023-07-12 19:10:46,216 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/dc1d1e60c2b94a1291973f788a370e95, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:10:46,217 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/d747715a45d847d59970893fa9599993 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/d747715a45d847d59970893fa9599993 2023-07-12 19:10:46,222 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/d747715a45d847d59970893fa9599993, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:10:46,222 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0e61b0eec6c8d265f5fb01657ce1d2a0 in 33ms, sequenceid=71, compaction requested=true 2023-07-12 19:10:46,223 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0e61b0eec6c8d265f5fb01657ce1d2a0: 2023-07-12 19:10:46,223 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:46,223 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:46,223 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:46,223 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,223 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:46,223 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:46,224 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,224 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:46,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:46,225 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,225 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:46,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:46,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:46,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:46,227 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 19:10:46,227 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@286f3d09[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:46,227 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 19:10:46,227 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,228 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,228 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,228 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,228 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:46,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,228 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:46,228 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,229 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:46,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,228 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:46,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,229 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:46,229 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:46,229 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:46,229 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:46,229 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:46,229 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:46,231 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:46,231 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,231 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,231 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:46,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:46,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:46,231 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:46,231 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,231 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:46,231 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:46,231 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,232 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:46,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:46,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:46,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:46,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:46,235 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:10:46,235 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 19:10:46,235 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:46,235 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:46,236 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,236 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:46,244 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,244 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,244 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,244 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,245 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,244 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:46,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:46,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,245 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:46,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,245 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,246 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,247 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:46,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:46,249 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,249 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,250 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,253 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,253 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,253 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,253 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,254 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,254 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,254 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,254 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,254 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,254 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:46,254 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:46,254 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,254 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,254 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,254 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,255 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,254 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:46,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,255 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,255 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:46,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,255 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,256 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,256 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,355 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:46,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:46,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:46,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:46,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,379 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:46,379 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,379 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 19:10:46,379 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 19:10:46,379 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,379 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,380 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,380 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,380 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,380 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,380 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,381 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:46,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,381 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,381 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:46,382 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,382 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:46,382 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:46,382 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,382 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:46,382 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:46,382 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,382 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,382 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,382 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:46,382 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:46,382 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:46,382 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:46,382 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,382 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:46,382 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:46,382 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:46,382 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:46,382 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:46,382 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:46,382 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:46,382 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. started... 2023-07-12 19:10:46,382 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0e61b0eec6c8d265f5fb01657ce1d2a0 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-07-12 19:10:46,383 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,383 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,383 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,383 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:46,384 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,384 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:46,394 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/8e92af53c5c34829b73d137ed716a284 2023-07-12 19:10:46,399 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8e92af53c5c34829b73d137ed716a284 2023-07-12 19:10:46,410 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/b5167621f55e4c98a7ac8af680e28581 2023-07-12 19:10:46,414 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5167621f55e4c98a7ac8af680e28581 2023-07-12 19:10:46,415 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/8e92af53c5c34829b73d137ed716a284 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/8e92af53c5c34829b73d137ed716a284 2023-07-12 19:10:46,419 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8e92af53c5c34829b73d137ed716a284 2023-07-12 19:10:46,419 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/8e92af53c5c34829b73d137ed716a284, entries=9, sequenceid=83, filesize=5.1 K 2023-07-12 19:10:46,420 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/b5167621f55e4c98a7ac8af680e28581 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/b5167621f55e4c98a7ac8af680e28581 2023-07-12 19:10:46,424 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5167621f55e4c98a7ac8af680e28581 2023-07-12 19:10:46,424 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/b5167621f55e4c98a7ac8af680e28581, entries=9, sequenceid=83, filesize=5.1 K 2023-07-12 19:10:46,425 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 0e61b0eec6c8d265f5fb01657ce1d2a0 in 43ms, sequenceid=83, compaction requested=true 2023-07-12 19:10:46,425 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0e61b0eec6c8d265f5fb01657ce1d2a0: 2023-07-12 19:10:46,425 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:46,425 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:46,425 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:46,425 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,425 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:46,425 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:46,427 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,427 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:46,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:46,427 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,427 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:46,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:46,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:46,429 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 19:10:46,429 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@191f46b0[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:46,429 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 19:10:46,429 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,429 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,430 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,429 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,430 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:46,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,430 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:46,430 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,430 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:46,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,430 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:46,430 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:46,430 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:46,430 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:46,430 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:46,430 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:46,430 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:46,432 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:46,432 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,432 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:46,432 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,432 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:46,432 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:46,432 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,432 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:46,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:46,432 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:46,432 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:46,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:46,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:46,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:46,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:46,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:10:46,436 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 19:10:46,436 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:46,436 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:46,436 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,436 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:46,437 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,437 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,437 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,437 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,437 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,437 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,437 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,437 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:46,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,438 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,438 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:46,440 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,441 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,441 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,443 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,443 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,443 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,443 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,443 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,443 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,443 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,443 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,444 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,444 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,444 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,444 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:46,444 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:46,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,443 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,444 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,444 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,443 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,444 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:46,444 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,444 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,444 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,444 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:46,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,545 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:46,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:46,564 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:46,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:46,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,567 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:46,567 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,567 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 19:10:46,567 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 19:10:46,567 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,568 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,569 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,569 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,569 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,569 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,570 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,570 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:46,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:46,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:46,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,571 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,571 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:46,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:46,571 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,571 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:46,571 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,571 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:46,571 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,571 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,571 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:46,571 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:46,571 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:46,572 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,572 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:46,572 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:46,571 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:46,571 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:46,572 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,572 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:46,572 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:46,572 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,572 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:46,572 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:46,572 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:46,572 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:46,572 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. started... 2023-07-12 19:10:46,573 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0e61b0eec6c8d265f5fb01657ce1d2a0 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-07-12 19:10:46,573 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,573 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,574 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,574 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:46,574 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,574 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:46,586 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/9e0699c405904b008f79ed9bd24ed491 2023-07-12 19:10:46,592 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9e0699c405904b008f79ed9bd24ed491 2023-07-12 19:10:46,602 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/9587e226eb2e4ea79e074d63b8088932 2023-07-12 19:10:46,608 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9587e226eb2e4ea79e074d63b8088932 2023-07-12 19:10:46,609 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/9e0699c405904b008f79ed9bd24ed491 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/9e0699c405904b008f79ed9bd24ed491 2023-07-12 19:10:46,615 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9e0699c405904b008f79ed9bd24ed491 2023-07-12 19:10:46,615 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/9e0699c405904b008f79ed9bd24ed491, entries=5, sequenceid=91, filesize=5.0 K 2023-07-12 19:10:46,617 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/9587e226eb2e4ea79e074d63b8088932 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/9587e226eb2e4ea79e074d63b8088932 2023-07-12 19:10:46,622 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9587e226eb2e4ea79e074d63b8088932 2023-07-12 19:10:46,623 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/9587e226eb2e4ea79e074d63b8088932, entries=5, sequenceid=91, filesize=5.0 K 2023-07-12 19:10:46,624 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 0e61b0eec6c8d265f5fb01657ce1d2a0 in 51ms, sequenceid=91, compaction requested=true 2023-07-12 19:10:46,624 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0e61b0eec6c8d265f5fb01657ce1d2a0: 2023-07-12 19:10:46,624 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:46,624 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:46,624 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:46,624 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,624 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:46,624 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:46,626 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,626 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:46,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:46,626 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,626 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:46,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:46,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:46,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:46,629 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 19:10:46,629 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a5a4e26[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:46,629 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 19:10:46,629 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,630 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,630 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,630 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,631 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:46,631 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:46,631 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:46,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,631 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,631 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:46,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,631 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:46,631 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:46,631 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:46,631 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:46,631 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:46,631 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:46,632 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:46,632 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,632 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:46,632 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,632 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:46,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:46,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:46,632 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,633 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:46,633 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,633 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:46,633 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:46,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:46,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:46,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:46,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:46,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:10:46,637 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 19:10:46,637 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:46,637 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:46,637 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,637 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:46,638 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,638 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,638 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,638 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,638 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,639 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,638 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:46,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:46,639 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:46,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,640 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,640 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:46,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,642 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:46,642 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,643 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,653 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,653 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,653 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,653 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,653 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,653 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,653 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:46,653 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:46,653 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,653 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,653 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,654 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:46,654 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,653 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,654 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,654 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:46,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,655 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,655 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,655 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,655 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,655 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,655 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,755 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:46,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:46,775 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:10:46,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:10:46,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,777 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:46,777 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,777 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 19:10:46,777 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 19:10:46,778 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,778 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,779 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,779 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,779 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,779 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,779 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,779 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:10:46,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:46,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,781 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:46,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:10:46,781 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:46,781 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,781 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,781 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:46,781 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,781 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:46,781 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,781 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,781 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:46,781 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:46,781 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:46,781 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 19:10:46,781 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:46,781 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,781 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,782 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:46,782 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 19:10:46,782 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:10:46,782 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 19:10:46,782 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:46,782 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:10:46,782 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:46,782 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:10:46,782 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:46,782 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. started... 2023-07-12 19:10:46,782 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0e61b0eec6c8d265f5fb01657ce1d2a0 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-07-12 19:10:46,783 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,783 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,783 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,783 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:46,783 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,783 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:46,790 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/bf0f657e1a804817b5b8e9e2d75bf42d 2023-07-12 19:10:46,793 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf0f657e1a804817b5b8e9e2d75bf42d 2023-07-12 19:10:46,801 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/95984a785fa4432aaa46b62e0ad439b2 2023-07-12 19:10:46,805 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 95984a785fa4432aaa46b62e0ad439b2 2023-07-12 19:10:46,806 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/bf0f657e1a804817b5b8e9e2d75bf42d as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/bf0f657e1a804817b5b8e9e2d75bf42d 2023-07-12 19:10:46,810 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf0f657e1a804817b5b8e9e2d75bf42d 2023-07-12 19:10:46,810 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/bf0f657e1a804817b5b8e9e2d75bf42d, entries=5, sequenceid=99, filesize=5.0 K 2023-07-12 19:10:46,811 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/95984a785fa4432aaa46b62e0ad439b2 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/95984a785fa4432aaa46b62e0ad439b2 2023-07-12 19:10:46,816 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 95984a785fa4432aaa46b62e0ad439b2 2023-07-12 19:10:46,816 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/95984a785fa4432aaa46b62e0ad439b2, entries=5, sequenceid=99, filesize=5.0 K 2023-07-12 19:10:46,817 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 0e61b0eec6c8d265f5fb01657ce1d2a0 in 35ms, sequenceid=99, compaction requested=true 2023-07-12 19:10:46,817 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0e61b0eec6c8d265f5fb01657ce1d2a0: 2023-07-12 19:10:46,818 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:46,818 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:10:46,818 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:10:46,818 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,818 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 19:10:46,818 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 19:10:46,819 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,819 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:46,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:46,820 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:46,820 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:10:46,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:46,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:46,822 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 19:10:46,822 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59b6a011[Count = 0] remaining members to acquire global barrier 2023-07-12 19:10:46,822 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 19:10:46,822 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,823 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,823 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,823 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,823 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,823 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:10:46,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,823 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:46,824 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:46,824 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:10:46,823 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:46,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,824 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:46,824 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:10:46,824 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 19:10:46,824 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 19:10:46,824 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:10:46,825 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,825 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:46,825 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,825 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:46,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:46,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:46,825 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:46,825 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,825 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:46,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:46,825 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 19:10:46,825 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,826 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 19:10:46,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:46,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:46,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:10:46,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:10:46,829 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 19:10:46,829 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:10:46,829 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:10:46,829 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,829 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:10:46,831 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,831 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,831 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,831 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,831 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,831 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:10:46,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,831 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,831 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:10:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,832 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:10:46,833 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:10:46,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,835 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:10:46,835 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,835 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,844 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,844 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,845 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,845 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,845 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:10:46,845 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:10:46,845 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,845 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,845 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 19:10:46,845 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:10:46,845 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:10:46,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:10:46,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:10:46,846 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:10:46,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,845 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,846 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:10:46,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:10:46,846 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:10:46,847 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,847 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:10:46,847 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,847 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:46,847 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,847 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,847 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:46,947 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:10:46,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:10:46,955 DEBUG [Listener at localhost.localdomain/40405] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,958 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:46,960 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:46,960 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:46,960 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:46,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0e61b0eec6c8d265f5fb01657ce1d2a0, UNASSIGN}] 2023-07-12 19:10:46,965 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0e61b0eec6c8d265f5fb01657ce1d2a0, UNASSIGN 2023-07-12 19:10:46,966 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=0e61b0eec6c8d265f5fb01657ce1d2a0, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:46,966 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1689189046966"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189046966"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189046966"}]},"ts":"1689189046966"} 2023-07-12 19:10:46,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 0e61b0eec6c8d265f5fb01657ce1d2a0, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:47,121 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 0e61b0eec6c8d265f5fb01657ce1d2a0, disabling compactions & flushes 2023-07-12 19:10:47,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:47,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:47,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. after waiting 0 ms 2023-07-12 19:10:47,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:47,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 0e61b0eec6c8d265f5fb01657ce1d2a0 2/2 column families, dataSize=50 B heapSize=720 B 2023-07-12 19:10:47,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/87d6d04db20f474ea2897ff4cae0719a 2023-07-12 19:10:47,144 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 87d6d04db20f474ea2897ff4cae0719a 2023-07-12 19:10:47,158 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/33fe1b8fcede4090a3b3e345bbc40189 2023-07-12 19:10:47,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 33fe1b8fcede4090a3b3e345bbc40189 2023-07-12 19:10:47,163 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf1/87d6d04db20f474ea2897ff4cae0719a as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/87d6d04db20f474ea2897ff4cae0719a 2023-07-12 19:10:47,167 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 87d6d04db20f474ea2897ff4cae0719a 2023-07-12 19:10:47,167 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/87d6d04db20f474ea2897ff4cae0719a, entries=1, sequenceid=103, filesize=4.9 K 2023-07-12 19:10:47,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.tmp/cf2/33fe1b8fcede4090a3b3e345bbc40189 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/33fe1b8fcede4090a3b3e345bbc40189 2023-07-12 19:10:47,173 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 33fe1b8fcede4090a3b3e345bbc40189 2023-07-12 19:10:47,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/33fe1b8fcede4090a3b3e345bbc40189, entries=1, sequenceid=103, filesize=4.9 K 2023-07-12 19:10:47,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 0e61b0eec6c8d265f5fb01657ce1d2a0 in 52ms, sequenceid=103, compaction requested=true 2023-07-12 19:10:47,179 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-07-12 19:10:47,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:47,180 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0. 2023-07-12 19:10:47,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 0e61b0eec6c8d265f5fb01657ce1d2a0: 2023-07-12 19:10:47,181 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,182 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=0e61b0eec6c8d265f5fb01657ce1d2a0, regionState=CLOSED 2023-07-12 19:10:47,182 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1689189047182"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189047182"}]},"ts":"1689189047182"} 2023-07-12 19:10:47,184 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-07-12 19:10:47,184 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 0e61b0eec6c8d265f5fb01657ce1d2a0, server=jenkins-hbase20.apache.org,38353,1689189020667 in 214 msec 2023-07-12 19:10:47,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-07-12 19:10:47,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0e61b0eec6c8d265f5fb01657ce1d2a0, UNASSIGN in 221 msec 2023-07-12 19:10:47,200 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=0e61b0eec6c8d265f5fb01657ce1d2a0, threads=16 2023-07-12 19:10:47,200 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/1b6e38155cce49ec9d7b8478b011d7f4 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,200 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/236c2e5d04814c849ee02ba7488bd884 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,200 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/33fe1b8fcede4090a3b3e345bbc40189 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,200 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/525cd0743cad400192a410acf27e8a5e for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,201 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/94277c2ac28b4a4a9d2f8a60045fcc25 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,201 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/9587e226eb2e4ea79e074d63b8088932 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,201 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/95984a785fa4432aaa46b62e0ad439b2 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,201 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/a0c5a22c1efb4dc99a241faa5a9b3080 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,201 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/b5167621f55e4c98a7ac8af680e28581 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,202 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/d747715a45d847d59970893fa9599993 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,202 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/45695b5932ee4511a54c041a405619f0 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,202 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/090cc5071bb041f69ad5258a25e1429d for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,202 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/72ccb17830a04baabfa21b59aad14e21 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,206 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,207 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/7a4f6744ef3248e6932fb626a8d2c353 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,207 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/87d6d04db20f474ea2897ff4cae0719a for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,212 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/525cd0743cad400192a410acf27e8a5e, top=false 2023-07-12 19:10:47,213 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/a0c5a22c1efb4dc99a241faa5a9b3080, top=false 2023-07-12 19:10:47,213 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/d747715a45d847d59970893fa9599993, top=false 2023-07-12 19:10:47,216 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5167621f55e4c98a7ac8af680e28581 2023-07-12 19:10:47,248 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 33fe1b8fcede4090a3b3e345bbc40189 2023-07-12 19:10:47,248 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/7a4f6744ef3248e6932fb626a8d2c353, top=false 2023-07-12 19:10:47,250 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 95984a785fa4432aaa46b62e0ad439b2 2023-07-12 19:10:47,250 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9587e226eb2e4ea79e074d63b8088932 2023-07-12 19:10:47,253 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 87d6d04db20f474ea2897ff4cae0719a 2023-07-12 19:10:47,253 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/45695b5932ee4511a54c041a405619f0, top=false 2023-07-12 19:10:47,259 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 33fe1b8fcede4090a3b3e345bbc40189 2023-07-12 19:10:47,259 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/33fe1b8fcede4090a3b3e345bbc40189, top=true 2023-07-12 19:10:47,260 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 95984a785fa4432aaa46b62e0ad439b2 2023-07-12 19:10:47,260 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/95984a785fa4432aaa46b62e0ad439b2, top=true 2023-07-12 19:10:47,264 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 87d6d04db20f474ea2897ff4cae0719a 2023-07-12 19:10:47,264 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/87d6d04db20f474ea2897ff4cae0719a, top=true 2023-07-12 19:10:47,282 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.splits/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-525cd0743cad400192a410acf27e8a5e for child: 6328ad5feb88f12be2101cce567df742, parent: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,283 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.splits/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-a0c5a22c1efb4dc99a241faa5a9b3080 for child: 6328ad5feb88f12be2101cce567df742, parent: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,285 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.splits/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-d747715a45d847d59970893fa9599993 for child: 6328ad5feb88f12be2101cce567df742, parent: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,285 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.splits/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-87d6d04db20f474ea2897ff4cae0719a for child: 22bf8485475c754fcd294e2642c4de0e, parent: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,285 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.splits/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-95984a785fa4432aaa46b62e0ad439b2 for child: 22bf8485475c754fcd294e2642c4de0e, parent: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,285 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/87d6d04db20f474ea2897ff4cae0719a for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,285 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/95984a785fa4432aaa46b62e0ad439b2 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,285 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/8e92af53c5c34829b73d137ed716a284 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,285 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/9e0699c405904b008f79ed9bd24ed491 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,286 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9587e226eb2e4ea79e074d63b8088932 2023-07-12 19:10:47,286 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/9587e226eb2e4ea79e074d63b8088932, top=true 2023-07-12 19:10:47,287 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.splits/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-33fe1b8fcede4090a3b3e345bbc40189 for child: 22bf8485475c754fcd294e2642c4de0e, parent: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,287 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/33fe1b8fcede4090a3b3e345bbc40189 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,287 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/bf0f657e1a804817b5b8e9e2d75bf42d for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,288 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/525cd0743cad400192a410acf27e8a5e for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,288 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/dc1d1e60c2b94a1291973f788a370e95 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,288 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/a0c5a22c1efb4dc99a241faa5a9b3080 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,290 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.splits/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-7a4f6744ef3248e6932fb626a8d2c353 for child: 6328ad5feb88f12be2101cce567df742, parent: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,293 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf0f657e1a804817b5b8e9e2d75bf42d 2023-07-12 19:10:47,294 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/dc1d1e60c2b94a1291973f788a370e95, top=false 2023-07-12 19:10:47,295 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/7a4f6744ef3248e6932fb626a8d2c353 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,298 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf0f657e1a804817b5b8e9e2d75bf42d 2023-07-12 19:10:47,298 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/bf0f657e1a804817b5b8e9e2d75bf42d, top=true 2023-07-12 19:10:47,298 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.splits/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-dc1d1e60c2b94a1291973f788a370e95 for child: 6328ad5feb88f12be2101cce567df742, parent: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,301 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/dc1d1e60c2b94a1291973f788a370e95 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,303 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8e92af53c5c34829b73d137ed716a284 2023-07-12 19:10:47,310 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.splits/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-45695b5932ee4511a54c041a405619f0 for child: 6328ad5feb88f12be2101cce567df742, parent: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,312 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9e0699c405904b008f79ed9bd24ed491 2023-07-12 19:10:47,312 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/d747715a45d847d59970893fa9599993 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,315 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/45695b5932ee4511a54c041a405619f0 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,317 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9e0699c405904b008f79ed9bd24ed491 2023-07-12 19:10:47,317 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/9e0699c405904b008f79ed9bd24ed491, top=true 2023-07-12 19:10:47,340 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5167621f55e4c98a7ac8af680e28581 2023-07-12 19:10:47,346 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.splits/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-bf0f657e1a804817b5b8e9e2d75bf42d for child: 22bf8485475c754fcd294e2642c4de0e, parent: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,347 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/bf0f657e1a804817b5b8e9e2d75bf42d for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,347 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.splits/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-9587e226eb2e4ea79e074d63b8088932 for child: 22bf8485475c754fcd294e2642c4de0e, parent: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,347 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/9587e226eb2e4ea79e074d63b8088932 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,369 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/.splits/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-9e0699c405904b008f79ed9bd24ed491 for child: 22bf8485475c754fcd294e2642c4de0e, parent: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,369 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/9e0699c405904b008f79ed9bd24ed491 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,370 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/94277c2ac28b4a4a9d2f8a60045fcc25 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,374 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8e92af53c5c34829b73d137ed716a284 2023-07-12 19:10:47,379 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/236c2e5d04814c849ee02ba7488bd884 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,379 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/1b6e38155cce49ec9d7b8478b011d7f4 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,386 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/b5167621f55e4c98a7ac8af680e28581 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,393 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/090cc5071bb041f69ad5258a25e1429d for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,394 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,401 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/8e92af53c5c34829b73d137ed716a284 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,401 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/72ccb17830a04baabfa21b59aad14e21 for region: 0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,401 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 0e61b0eec6c8d265f5fb01657ce1d2a0 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 19:10:47,436 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-07-12 19:10:47,438 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-07-12 19:10:47,440 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1689189047440"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1689189047440"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1689189047440"}]},"ts":"1689189047440"} 2023-07-12 19:10:47,440 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689189047440"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189047440"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189047440"}]},"ts":"1689189047440"} 2023-07-12 19:10:47,440 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689189047440"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189047440"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189047440"}]},"ts":"1689189047440"} 2023-07-12 19:10:47,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6328ad5feb88f12be2101cce567df742, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=22bf8485475c754fcd294e2642c4de0e, ASSIGN}] 2023-07-12 19:10:47,444 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=22bf8485475c754fcd294e2642c4de0e, ASSIGN 2023-07-12 19:10:47,445 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=22bf8485475c754fcd294e2642c4de0e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,38353,1689189020667; forceNewPlan=false, retain=false 2023-07-12 19:10:47,446 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6328ad5feb88f12be2101cce567df742, ASSIGN 2023-07-12 19:10:47,446 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6328ad5feb88f12be2101cce567df742, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,38353,1689189020667; forceNewPlan=false, retain=false 2023-07-12 19:10:47,595 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-12 19:10:47,599 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=22bf8485475c754fcd294e2642c4de0e, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:47,600 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689189047599"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189047599"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189047599"}]},"ts":"1689189047599"} 2023-07-12 19:10:47,603 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; OpenRegionProcedure 22bf8485475c754fcd294e2642c4de0e, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:47,605 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=6328ad5feb88f12be2101cce567df742, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:47,605 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689189047605"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189047605"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189047605"}]},"ts":"1689189047605"} 2023-07-12 19:10:47,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=75, state=RUNNABLE; OpenRegionProcedure 6328ad5feb88f12be2101cce567df742, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:47,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. 2023-07-12 19:10:47,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6328ad5feb88f12be2101cce567df742, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.', STARTKEY => '', ENDKEY => '60'} 2023-07-12 19:10:47,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:47,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 6328ad5feb88f12be2101cce567df742 2023-07-12 19:10:47,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:47,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 6328ad5feb88f12be2101cce567df742 2023-07-12 19:10:47,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 6328ad5feb88f12be2101cce567df742 2023-07-12 19:10:47,764 INFO [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6328ad5feb88f12be2101cce567df742 2023-07-12 19:10:47,766 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1 2023-07-12 19:10:47,766 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1 2023-07-12 19:10:47,766 INFO [StoreOpener-6328ad5feb88f12be2101cce567df742-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6328ad5feb88f12be2101cce567df742 columnFamilyName cf1 2023-07-12 19:10:47,782 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/090cc5071bb041f69ad5258a25e1429d.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/090cc5071bb041f69ad5258a25e1429d-bottom 2023-07-12 19:10:47,786 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/72ccb17830a04baabfa21b59aad14e21.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/72ccb17830a04baabfa21b59aad14e21-bottom 2023-07-12 19:10:47,790 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f-bottom 2023-07-12 19:10:47,794 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8e92af53c5c34829b73d137ed716a284.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,795 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/8e92af53c5c34829b73d137ed716a284.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/8e92af53c5c34829b73d137ed716a284-bottom 2023-07-12 19:10:47,798 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-45695b5932ee4511a54c041a405619f0 2023-07-12 19:10:47,801 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-7a4f6744ef3248e6932fb626a8d2c353 2023-07-12 19:10:47,804 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-dc1d1e60c2b94a1291973f788a370e95 2023-07-12 19:10:47,805 INFO [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(310): Store=6328ad5feb88f12be2101cce567df742/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:47,805 INFO [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6328ad5feb88f12be2101cce567df742 2023-07-12 19:10:47,806 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2 2023-07-12 19:10:47,806 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2 2023-07-12 19:10:47,806 INFO [StoreOpener-6328ad5feb88f12be2101cce567df742-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6328ad5feb88f12be2101cce567df742 columnFamilyName cf2 2023-07-12 19:10:47,816 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/1b6e38155cce49ec9d7b8478b011d7f4.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/1b6e38155cce49ec9d7b8478b011d7f4-bottom 2023-07-12 19:10:47,820 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/236c2e5d04814c849ee02ba7488bd884.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/236c2e5d04814c849ee02ba7488bd884-bottom 2023-07-12 19:10:47,824 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/94277c2ac28b4a4a9d2f8a60045fcc25.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/94277c2ac28b4a4a9d2f8a60045fcc25-bottom 2023-07-12 19:10:47,826 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5167621f55e4c98a7ac8af680e28581.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,827 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/b5167621f55e4c98a7ac8af680e28581.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/b5167621f55e4c98a7ac8af680e28581-bottom 2023-07-12 19:10:47,830 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-525cd0743cad400192a410acf27e8a5e 2023-07-12 19:10:47,834 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-a0c5a22c1efb4dc99a241faa5a9b3080 2023-07-12 19:10:47,837 DEBUG [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-d747715a45d847d59970893fa9599993 2023-07-12 19:10:47,837 INFO [StoreOpener-6328ad5feb88f12be2101cce567df742-1] regionserver.HStore(310): Store=6328ad5feb88f12be2101cce567df742/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:47,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742 2023-07-12 19:10:47,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742 2023-07-12 19:10:47,840 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:10:47,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 6328ad5feb88f12be2101cce567df742 2023-07-12 19:10:47,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 6328ad5feb88f12be2101cce567df742; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63573432, jitterRate=-0.05268204212188721}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:10:47,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 6328ad5feb88f12be2101cce567df742: 2023-07-12 19:10:47,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742., pid=78, masterSystemTime=1689189047755 2023-07-12 19:10:47,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:10:47,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:10:47,843 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:10:47,843 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:10:47,843 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. 2023-07-12 19:10:47,843 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. 2023-07-12 19:10:47,843 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1912): 6328ad5feb88f12be2101cce567df742/cf2 is initiating minor compaction (all files) 2023-07-12 19:10:47,843 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1912): 6328ad5feb88f12be2101cce567df742/cf1 is initiating minor compaction (all files) 2023-07-12 19:10:47,843 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 6328ad5feb88f12be2101cce567df742/cf2 in testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. 2023-07-12 19:10:47,843 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6328ad5feb88f12be2101cce567df742/cf1 in testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. 2023-07-12 19:10:47,843 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/236c2e5d04814c849ee02ba7488bd884.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/236c2e5d04814c849ee02ba7488bd884-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/94277c2ac28b4a4a9d2f8a60045fcc25.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/94277c2ac28b4a4a9d2f8a60045fcc25-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/1b6e38155cce49ec9d7b8478b011d7f4.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/1b6e38155cce49ec9d7b8478b011d7f4-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-525cd0743cad400192a410acf27e8a5e, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-a0c5a22c1efb4dc99a241faa5a9b3080, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-d747715a45d847d59970893fa9599993, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/b5167621f55e4c98a7ac8af680e28581.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/b5167621f55e4c98a7ac8af680e28581-bottom] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/.tmp, totalSize=35.0 K 2023-07-12 19:10:47,843 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/090cc5071bb041f69ad5258a25e1429d.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/090cc5071bb041f69ad5258a25e1429d-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/72ccb17830a04baabfa21b59aad14e21.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/72ccb17830a04baabfa21b59aad14e21-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-7a4f6744ef3248e6932fb626a8d2c353, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-45695b5932ee4511a54c041a405619f0, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-dc1d1e60c2b94a1291973f788a370e95, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/8e92af53c5c34829b73d137ed716a284.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/8e92af53c5c34829b73d137ed716a284-bottom] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/.tmp, totalSize=35.0 K 2023-07-12 19:10:47,844 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 236c2e5d04814c849ee02ba7488bd884.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689189045215 2023-07-12 19:10:47,844 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 090cc5071bb041f69ad5258a25e1429d.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689189045215 2023-07-12 19:10:47,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. 2023-07-12 19:10:47,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. 2023-07-12 19:10:47,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. 2023-07-12 19:10:47,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 22bf8485475c754fcd294e2642c4de0e, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.', STARTKEY => '60', ENDKEY => ''} 2023-07-12 19:10:47,845 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 94277c2ac28b4a4a9d2f8a60045fcc25.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689189045406 2023-07-12 19:10:47,845 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 72ccb17830a04baabfa21b59aad14e21.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689189045406 2023-07-12 19:10:47,845 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 1b6e38155cce49ec9d7b8478b011d7f4.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689189045584 2023-07-12 19:10:47,845 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 76a1a1a0f0fe4ea49628e7aa53b01b0f.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689189045584 2023-07-12 19:10:47,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:47,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:47,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:47,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:47,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:47,846 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=6328ad5feb88f12be2101cce567df742, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:47,846 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689189047845"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189047845"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189047845"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189047845"}]},"ts":"1689189047845"} 2023-07-12 19:10:47,846 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-525cd0743cad400192a410acf27e8a5e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689189045797 2023-07-12 19:10:47,846 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-7a4f6744ef3248e6932fb626a8d2c353, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689189045797 2023-07-12 19:10:47,846 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-a0c5a22c1efb4dc99a241faa5a9b3080, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689189045978 2023-07-12 19:10:47,846 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-45695b5932ee4511a54c041a405619f0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689189045978 2023-07-12 19:10:47,847 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-dc1d1e60c2b94a1291973f788a370e95, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689189046172 2023-07-12 19:10:47,847 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 8e92af53c5c34829b73d137ed716a284.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689189046360 2023-07-12 19:10:47,847 INFO [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:47,847 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-d747715a45d847d59970893fa9599993, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689189046172 2023-07-12 19:10:47,848 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting b5167621f55e4c98a7ac8af680e28581.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689189046360 2023-07-12 19:10:47,848 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1 2023-07-12 19:10:47,848 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1 2023-07-12 19:10:47,849 INFO [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 22bf8485475c754fcd294e2642c4de0e columnFamilyName cf1 2023-07-12 19:10:47,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=75 2023-07-12 19:10:47,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=75, state=SUCCESS; OpenRegionProcedure 6328ad5feb88f12be2101cce567df742, server=jenkins-hbase20.apache.org,38353,1689189020667 in 241 msec 2023-07-12 19:10:47,852 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6328ad5feb88f12be2101cce567df742, ASSIGN in 406 msec 2023-07-12 19:10:47,859 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 19:10:47,868 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/090cc5071bb041f69ad5258a25e1429d.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/090cc5071bb041f69ad5258a25e1429d-top 2023-07-12 19:10:47,869 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6328ad5feb88f12be2101cce567df742#cf1#compaction#72 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:47,871 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] throttle.PressureAwareThroughputController(145): 6328ad5feb88f12be2101cce567df742#cf2#compaction#73 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:47,873 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/72ccb17830a04baabfa21b59aad14e21.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/72ccb17830a04baabfa21b59aad14e21-top 2023-07-12 19:10:47,884 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f-top 2023-07-12 19:10:47,889 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8e92af53c5c34829b73d137ed716a284.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,889 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/8e92af53c5c34829b73d137ed716a284.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/8e92af53c5c34829b73d137ed716a284-top 2023-07-12 19:10:47,894 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 87d6d04db20f474ea2897ff4cae0719a 2023-07-12 19:10:47,894 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-87d6d04db20f474ea2897ff4cae0719a 2023-07-12 19:10:47,899 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/.tmp/cf1/a29e3bc0fc5a48569a319f0d7b561562 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/a29e3bc0fc5a48569a319f0d7b561562 2023-07-12 19:10:47,900 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9e0699c405904b008f79ed9bd24ed491 2023-07-12 19:10:47,900 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-9e0699c405904b008f79ed9bd24ed491 2023-07-12 19:10:47,904 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/.tmp/cf2/0284cd04ff8e487aac57f891967b50c0 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/0284cd04ff8e487aac57f891967b50c0 2023-07-12 19:10:47,905 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6328ad5feb88f12be2101cce567df742/cf1 of 6328ad5feb88f12be2101cce567df742 into a29e3bc0fc5a48569a319f0d7b561562(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:47,905 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6328ad5feb88f12be2101cce567df742: 2023-07-12 19:10:47,905 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742., storeName=6328ad5feb88f12be2101cce567df742/cf1, priority=9, startTime=1689189047842; duration=0sec 2023-07-12 19:10:47,906 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:47,907 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf0f657e1a804817b5b8e9e2d75bf42d 2023-07-12 19:10:47,907 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-bf0f657e1a804817b5b8e9e2d75bf42d 2023-07-12 19:10:47,909 INFO [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(310): Store=22bf8485475c754fcd294e2642c4de0e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:47,909 INFO [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:47,910 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6328ad5feb88f12be2101cce567df742/cf2 of 6328ad5feb88f12be2101cce567df742 into 0284cd04ff8e487aac57f891967b50c0(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:47,910 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6328ad5feb88f12be2101cce567df742: 2023-07-12 19:10:47,910 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742., storeName=6328ad5feb88f12be2101cce567df742/cf2, priority=9, startTime=1689189047843; duration=0sec 2023-07-12 19:10:47,911 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:47,911 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2 2023-07-12 19:10:47,911 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2 2023-07-12 19:10:47,911 INFO [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 22bf8485475c754fcd294e2642c4de0e columnFamilyName cf2 2023-07-12 19:10:47,921 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/1b6e38155cce49ec9d7b8478b011d7f4.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/1b6e38155cce49ec9d7b8478b011d7f4-top 2023-07-12 19:10:47,924 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/236c2e5d04814c849ee02ba7488bd884.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/236c2e5d04814c849ee02ba7488bd884-top 2023-07-12 19:10:47,927 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/94277c2ac28b4a4a9d2f8a60045fcc25.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/94277c2ac28b4a4a9d2f8a60045fcc25-top 2023-07-12 19:10:47,930 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5167621f55e4c98a7ac8af680e28581.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:47,930 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/b5167621f55e4c98a7ac8af680e28581.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/b5167621f55e4c98a7ac8af680e28581-top 2023-07-12 19:10:47,935 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 33fe1b8fcede4090a3b3e345bbc40189 2023-07-12 19:10:47,935 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-33fe1b8fcede4090a3b3e345bbc40189 2023-07-12 19:10:47,938 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9587e226eb2e4ea79e074d63b8088932 2023-07-12 19:10:47,938 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-9587e226eb2e4ea79e074d63b8088932 2023-07-12 19:10:47,942 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 95984a785fa4432aaa46b62e0ad439b2 2023-07-12 19:10:47,942 DEBUG [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-95984a785fa4432aaa46b62e0ad439b2 2023-07-12 19:10:47,942 INFO [StoreOpener-22bf8485475c754fcd294e2642c4de0e-1] regionserver.HStore(310): Store=22bf8485475c754fcd294e2642c4de0e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:47,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:47,944 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:47,945 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:10:47,946 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:47,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 22bf8485475c754fcd294e2642c4de0e; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64095431, jitterRate=-0.044903650879859924}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:10:47,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 22bf8485475c754fcd294e2642c4de0e: 2023-07-12 19:10:47,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e., pid=77, masterSystemTime=1689189047755 2023-07-12 19:10:47,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:10:47,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:10:47,948 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:10:47,948 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:10:47,948 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. 2023-07-12 19:10:47,949 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. 2023-07-12 19:10:47,949 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1912): 22bf8485475c754fcd294e2642c4de0e/cf2 is initiating minor compaction (all files) 2023-07-12 19:10:47,949 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1912): 22bf8485475c754fcd294e2642c4de0e/cf1 is initiating minor compaction (all files) 2023-07-12 19:10:47,949 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 22bf8485475c754fcd294e2642c4de0e/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. 2023-07-12 19:10:47,949 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 22bf8485475c754fcd294e2642c4de0e/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. 2023-07-12 19:10:47,949 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/090cc5071bb041f69ad5258a25e1429d.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/090cc5071bb041f69ad5258a25e1429d-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/72ccb17830a04baabfa21b59aad14e21.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/72ccb17830a04baabfa21b59aad14e21-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/8e92af53c5c34829b73d137ed716a284.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/8e92af53c5c34829b73d137ed716a284-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-9e0699c405904b008f79ed9bd24ed491, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-bf0f657e1a804817b5b8e9e2d75bf42d, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-87d6d04db20f474ea2897ff4cae0719a] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/.tmp, totalSize=34.8 K 2023-07-12 19:10:47,949 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/236c2e5d04814c849ee02ba7488bd884.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/236c2e5d04814c849ee02ba7488bd884-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/94277c2ac28b4a4a9d2f8a60045fcc25.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/94277c2ac28b4a4a9d2f8a60045fcc25-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/1b6e38155cce49ec9d7b8478b011d7f4.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/1b6e38155cce49ec9d7b8478b011d7f4-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/b5167621f55e4c98a7ac8af680e28581.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/b5167621f55e4c98a7ac8af680e28581-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-9587e226eb2e4ea79e074d63b8088932, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-95984a785fa4432aaa46b62e0ad439b2, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-33fe1b8fcede4090a3b3e345bbc40189] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/.tmp, totalSize=34.8 K 2023-07-12 19:10:47,949 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 090cc5071bb041f69ad5258a25e1429d.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689189045215 2023-07-12 19:10:47,949 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 236c2e5d04814c849ee02ba7488bd884.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689189045215 2023-07-12 19:10:47,950 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. 2023-07-12 19:10:47,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. 2023-07-12 19:10:47,950 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 72ccb17830a04baabfa21b59aad14e21.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689189045406 2023-07-12 19:10:47,950 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 94277c2ac28b4a4a9d2f8a60045fcc25.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689189045406 2023-07-12 19:10:47,950 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=22bf8485475c754fcd294e2642c4de0e, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:47,950 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 76a1a1a0f0fe4ea49628e7aa53b01b0f.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689189045584 2023-07-12 19:10:47,950 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 1b6e38155cce49ec9d7b8478b011d7f4.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689189045584 2023-07-12 19:10:47,950 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689189047950"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189047950"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189047950"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189047950"}]},"ts":"1689189047950"} 2023-07-12 19:10:47,950 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 8e92af53c5c34829b73d137ed716a284.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1689189046360 2023-07-12 19:10:47,950 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting b5167621f55e4c98a7ac8af680e28581.0e61b0eec6c8d265f5fb01657ce1d2a0, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1689189046360 2023-07-12 19:10:47,951 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-9e0699c405904b008f79ed9bd24ed491, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-07-12 19:10:47,951 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-9587e226eb2e4ea79e074d63b8088932, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-07-12 19:10:47,951 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-bf0f657e1a804817b5b8e9e2d75bf42d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-07-12 19:10:47,951 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-95984a785fa4432aaa46b62e0ad439b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-07-12 19:10:47,951 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-87d6d04db20f474ea2897ff4cae0719a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-07-12 19:10:47,951 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-33fe1b8fcede4090a3b3e345bbc40189, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-07-12 19:10:47,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-07-12 19:10:47,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; OpenRegionProcedure 22bf8485475c754fcd294e2642c4de0e, server=jenkins-hbase20.apache.org,38353,1689189020667 in 348 msec 2023-07-12 19:10:47,954 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-07-12 19:10:47,954 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=22bf8485475c754fcd294e2642c4de0e, ASSIGN in 510 msec 2023-07-12 19:10:47,956 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e in 998 msec 2023-07-12 19:10:47,967 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] throttle.PressureAwareThroughputController(145): 22bf8485475c754fcd294e2642c4de0e#cf2#compaction#75 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:47,967 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] throttle.PressureAwareThroughputController(145): 22bf8485475c754fcd294e2642c4de0e#cf1#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:47,976 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 19:10:47,980 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52086, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 19:10:47,980 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38353] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. 2023-07-12 19:10:47,981 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38353] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-07-12 19:10:47,981 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38353] regionserver.HStore(1912): 6328ad5feb88f12be2101cce567df742/cf1 is initiating major compaction (all files) 2023-07-12 19:10:47,981 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38353] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@25ce5d50; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:10:47,981 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38353] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-07-12 19:10:47,981 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38353] regionserver.HStore(1912): 6328ad5feb88f12be2101cce567df742/cf2 is initiating major compaction (all files) 2023-07-12 19:10:47,981 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38353] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@39aa3dd2; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-07-12 19:10:47,983 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38353] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. 2023-07-12 19:10:47,983 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38353] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 7 compacting, 0 eligible, 16 blocking 2023-07-12 19:10:47,983 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38353] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-07-12 19:10:47,983 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38353] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-07-12 19:10:47,983 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38353] regionserver.CompactSplit(415): Not compacting testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. because compaction request was cancelled 2023-07-12 19:10:47,983 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38353] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 7 compacting, 0 eligible, 16 blocking 2023-07-12 19:10:47,983 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38353] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-07-12 19:10:47,984 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38353] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-07-12 19:10:47,984 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38353] regionserver.CompactSplit(415): Not compacting testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. because compaction request was cancelled 2023-07-12 19:10:47,984 INFO [Listener at localhost.localdomain/40405] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-07-12 19:10:47,992 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for d56d65f62cb04298ae17e3520243bbc2: NONE, but ROW specified in column family configuration 2023-07-12 19:10:47,992 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 943105fc4a8c45d797d5d95f66c14a9f: NONE, but ROW specified in column family configuration 2023-07-12 19:10:47,993 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/.tmp/cf1/d56d65f62cb04298ae17e3520243bbc2 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/d56d65f62cb04298ae17e3520243bbc2 2023-07-12 19:10:47,993 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/.tmp/cf2/943105fc4a8c45d797d5d95f66c14a9f as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/943105fc4a8c45d797d5d95f66c14a9f 2023-07-12 19:10:47,995 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38353] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. is not mergeable because it has references 2023-07-12 19:10:47,996 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38353] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. is not mergeable because it has references 2023-07-12 19:10:47,997 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for d56d65f62cb04298ae17e3520243bbc2: NONE, but ROW specified in column family configuration 2023-07-12 19:10:47,997 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 943105fc4a8c45d797d5d95f66c14a9f: NONE, but ROW specified in column family configuration 2023-07-12 19:10:47,998 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 22bf8485475c754fcd294e2642c4de0e/cf1 of 22bf8485475c754fcd294e2642c4de0e into d56d65f62cb04298ae17e3520243bbc2(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:47,998 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 22bf8485475c754fcd294e2642c4de0e/cf2 of 22bf8485475c754fcd294e2642c4de0e into 943105fc4a8c45d797d5d95f66c14a9f(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:47,998 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 22bf8485475c754fcd294e2642c4de0e: 2023-07-12 19:10:47,998 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 22bf8485475c754fcd294e2642c4de0e: 2023-07-12 19:10:47,998 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e., storeName=22bf8485475c754fcd294e2642c4de0e/cf1, priority=9, startTime=1689189047948; duration=0sec 2023-07-12 19:10:47,998 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e., storeName=22bf8485475c754fcd294e2642c4de0e/cf2, priority=9, startTime=1689189047948; duration=0sec 2023-07-12 19:10:47,998 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-07-12 19:10:47,998 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-07-12 19:10:47,998 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6328ad5feb88f12be2101cce567df742/cf1 in testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. 2023-07-12 19:10:47,998 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 6328ad5feb88f12be2101cce567df742/cf2 in testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. 2023-07-12 19:10:47,998 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/a29e3bc0fc5a48569a319f0d7b561562] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/.tmp, totalSize=6.8 K 2023-07-12 19:10:47,998 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/0284cd04ff8e487aac57f891967b50c0] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/.tmp, totalSize=6.8 K 2023-07-12 19:10:47,998 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting a29e3bc0fc5a48569a319f0d7b561562, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689189045215 2023-07-12 19:10:47,999 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 0284cd04ff8e487aac57f891967b50c0, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689189045215 2023-07-12 19:10:48,002 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6328ad5feb88f12be2101cce567df742#cf1#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:48,002 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] throttle.PressureAwareThroughputController(145): 6328ad5feb88f12be2101cce567df742#cf2#compaction#77 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:10:48,013 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/.tmp/cf1/86a85ae2d0e144debe53d815234947bb as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/86a85ae2d0e144debe53d815234947bb 2023-07-12 19:10:48,013 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/.tmp/cf2/6f1432cdb469471dab3732646420e30a as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/6f1432cdb469471dab3732646420e30a 2023-07-12 19:10:48,017 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 6328ad5feb88f12be2101cce567df742/cf2 of 6328ad5feb88f12be2101cce567df742 into 6f1432cdb469471dab3732646420e30a(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:48,017 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 6328ad5feb88f12be2101cce567df742/cf1 of 6328ad5feb88f12be2101cce567df742 into 86a85ae2d0e144debe53d815234947bb(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:10:48,017 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6328ad5feb88f12be2101cce567df742: 2023-07-12 19:10:48,017 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6328ad5feb88f12be2101cce567df742: 2023-07-12 19:10:48,019 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1689189047981; duration=0sec 2023-07-12 19:10:48,019 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1689189047981; duration=0sec 2023-07-12 19:10:48,019 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:48,019 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:10:48,112 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38353] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. is not mergeable because it has references 2023-07-12 19:10:48,113 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38353] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. is not mergeable because it has references 2023-07-12 19:10:48,118 DEBUG [Listener at localhost.localdomain/40405] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-07-12 19:10:48,120 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:10:48,122 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:48,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:48,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:48,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-12 19:10:48,126 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189048125"}]},"ts":"1689189048125"} 2023-07-12 19:10:48,127 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-07-12 19:10:48,128 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-07-12 19:10:48,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6328ad5feb88f12be2101cce567df742, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=22bf8485475c754fcd294e2642c4de0e, UNASSIGN}] 2023-07-12 19:10:48,129 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=22bf8485475c754fcd294e2642c4de0e, UNASSIGN 2023-07-12 19:10:48,130 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=22bf8485475c754fcd294e2642c4de0e, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:48,130 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689189048130"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189048130"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189048130"}]},"ts":"1689189048130"} 2023-07-12 19:10:48,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 22bf8485475c754fcd294e2642c4de0e, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:48,133 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6328ad5feb88f12be2101cce567df742, UNASSIGN 2023-07-12 19:10:48,134 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=6328ad5feb88f12be2101cce567df742, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:48,134 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689189048134"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189048134"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189048134"}]},"ts":"1689189048134"} 2023-07-12 19:10:48,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 6328ad5feb88f12be2101cce567df742, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:48,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-12 19:10:48,284 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 6328ad5feb88f12be2101cce567df742 2023-07-12 19:10:48,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 6328ad5feb88f12be2101cce567df742, disabling compactions & flushes 2023-07-12 19:10:48,284 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. 2023-07-12 19:10:48,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. 2023-07-12 19:10:48,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. after waiting 0 ms 2023-07-12 19:10:48,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. 2023-07-12 19:10:48,286 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/090cc5071bb041f69ad5258a25e1429d.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/090cc5071bb041f69ad5258a25e1429d-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/72ccb17830a04baabfa21b59aad14e21.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/72ccb17830a04baabfa21b59aad14e21-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-7a4f6744ef3248e6932fb626a8d2c353, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-45695b5932ee4511a54c041a405619f0, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-dc1d1e60c2b94a1291973f788a370e95, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/a29e3bc0fc5a48569a319f0d7b561562, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/8e92af53c5c34829b73d137ed716a284.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/8e92af53c5c34829b73d137ed716a284-bottom] to archive 2023-07-12 19:10:48,288 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:10:48,294 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/090cc5071bb041f69ad5258a25e1429d.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/090cc5071bb041f69ad5258a25e1429d.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,296 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/72ccb17830a04baabfa21b59aad14e21.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/72ccb17830a04baabfa21b59aad14e21.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,298 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,300 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-7a4f6744ef3248e6932fb626a8d2c353 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-7a4f6744ef3248e6932fb626a8d2c353 2023-07-12 19:10:48,301 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-45695b5932ee4511a54c041a405619f0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-45695b5932ee4511a54c041a405619f0 2023-07-12 19:10:48,303 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-dc1d1e60c2b94a1291973f788a370e95 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-dc1d1e60c2b94a1291973f788a370e95 2023-07-12 19:10:48,304 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/a29e3bc0fc5a48569a319f0d7b561562 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/a29e3bc0fc5a48569a319f0d7b561562 2023-07-12 19:10:48,306 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/8e92af53c5c34829b73d137ed716a284.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/8e92af53c5c34829b73d137ed716a284.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,308 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/236c2e5d04814c849ee02ba7488bd884.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/236c2e5d04814c849ee02ba7488bd884-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/94277c2ac28b4a4a9d2f8a60045fcc25.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/94277c2ac28b4a4a9d2f8a60045fcc25-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/1b6e38155cce49ec9d7b8478b011d7f4.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/1b6e38155cce49ec9d7b8478b011d7f4-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-525cd0743cad400192a410acf27e8a5e, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-a0c5a22c1efb4dc99a241faa5a9b3080, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-d747715a45d847d59970893fa9599993, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/0284cd04ff8e487aac57f891967b50c0, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/b5167621f55e4c98a7ac8af680e28581.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/b5167621f55e4c98a7ac8af680e28581-bottom] to archive 2023-07-12 19:10:48,308 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:10:48,310 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/236c2e5d04814c849ee02ba7488bd884.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/236c2e5d04814c849ee02ba7488bd884.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,311 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/94277c2ac28b4a4a9d2f8a60045fcc25.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/94277c2ac28b4a4a9d2f8a60045fcc25.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,312 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/1b6e38155cce49ec9d7b8478b011d7f4.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/1b6e38155cce49ec9d7b8478b011d7f4.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,314 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-525cd0743cad400192a410acf27e8a5e to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-525cd0743cad400192a410acf27e8a5e 2023-07-12 19:10:48,315 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-a0c5a22c1efb4dc99a241faa5a9b3080 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-a0c5a22c1efb4dc99a241faa5a9b3080 2023-07-12 19:10:48,316 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-d747715a45d847d59970893fa9599993 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-d747715a45d847d59970893fa9599993 2023-07-12 19:10:48,317 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/0284cd04ff8e487aac57f891967b50c0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/0284cd04ff8e487aac57f891967b50c0 2023-07-12 19:10:48,318 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/b5167621f55e4c98a7ac8af680e28581.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/b5167621f55e4c98a7ac8af680e28581.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,322 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-07-12 19:10:48,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:48,324 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742. 2023-07-12 19:10:48,324 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 6328ad5feb88f12be2101cce567df742: 2023-07-12 19:10:48,325 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 6328ad5feb88f12be2101cce567df742 2023-07-12 19:10:48,325 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:48,325 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 22bf8485475c754fcd294e2642c4de0e, disabling compactions & flushes 2023-07-12 19:10:48,326 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. 2023-07-12 19:10:48,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. 2023-07-12 19:10:48,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. after waiting 0 ms 2023-07-12 19:10:48,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. 2023-07-12 19:10:48,326 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=6328ad5feb88f12be2101cce567df742, regionState=CLOSED 2023-07-12 19:10:48,326 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689189048326"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189048326"}]},"ts":"1689189048326"} 2023-07-12 19:10:48,327 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/090cc5071bb041f69ad5258a25e1429d.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/090cc5071bb041f69ad5258a25e1429d-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/72ccb17830a04baabfa21b59aad14e21.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/72ccb17830a04baabfa21b59aad14e21-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/8e92af53c5c34829b73d137ed716a284.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/8e92af53c5c34829b73d137ed716a284-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-9e0699c405904b008f79ed9bd24ed491, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-bf0f657e1a804817b5b8e9e2d75bf42d, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-87d6d04db20f474ea2897ff4cae0719a] to archive 2023-07-12 19:10:48,328 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:10:48,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-07-12 19:10:48,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 6328ad5feb88f12be2101cce567df742, server=jenkins-hbase20.apache.org,38353,1689189020667 in 192 msec 2023-07-12 19:10:48,330 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/090cc5071bb041f69ad5258a25e1429d.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/090cc5071bb041f69ad5258a25e1429d.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6328ad5feb88f12be2101cce567df742, UNASSIGN in 200 msec 2023-07-12 19:10:48,332 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/72ccb17830a04baabfa21b59aad14e21.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/72ccb17830a04baabfa21b59aad14e21.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,333 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,335 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/8e92af53c5c34829b73d137ed716a284.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/8e92af53c5c34829b73d137ed716a284.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,337 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-9e0699c405904b008f79ed9bd24ed491 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-9e0699c405904b008f79ed9bd24ed491 2023-07-12 19:10:48,338 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-bf0f657e1a804817b5b8e9e2d75bf42d to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-bf0f657e1a804817b5b8e9e2d75bf42d 2023-07-12 19:10:48,339 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-87d6d04db20f474ea2897ff4cae0719a to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-87d6d04db20f474ea2897ff4cae0719a 2023-07-12 19:10:48,341 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/236c2e5d04814c849ee02ba7488bd884.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/236c2e5d04814c849ee02ba7488bd884-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/94277c2ac28b4a4a9d2f8a60045fcc25.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/94277c2ac28b4a4a9d2f8a60045fcc25-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/1b6e38155cce49ec9d7b8478b011d7f4.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/1b6e38155cce49ec9d7b8478b011d7f4-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/b5167621f55e4c98a7ac8af680e28581.0e61b0eec6c8d265f5fb01657ce1d2a0->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/b5167621f55e4c98a7ac8af680e28581-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-9587e226eb2e4ea79e074d63b8088932, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-95984a785fa4432aaa46b62e0ad439b2, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-33fe1b8fcede4090a3b3e345bbc40189] to archive 2023-07-12 19:10:48,341 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:10:48,342 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/236c2e5d04814c849ee02ba7488bd884.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/236c2e5d04814c849ee02ba7488bd884.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,343 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/94277c2ac28b4a4a9d2f8a60045fcc25.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/94277c2ac28b4a4a9d2f8a60045fcc25.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,344 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/1b6e38155cce49ec9d7b8478b011d7f4.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/1b6e38155cce49ec9d7b8478b011d7f4.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,345 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/b5167621f55e4c98a7ac8af680e28581.0e61b0eec6c8d265f5fb01657ce1d2a0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/b5167621f55e4c98a7ac8af680e28581.0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,345 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-9587e226eb2e4ea79e074d63b8088932 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-9587e226eb2e4ea79e074d63b8088932 2023-07-12 19:10:48,346 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-95984a785fa4432aaa46b62e0ad439b2 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-95984a785fa4432aaa46b62e0ad439b2 2023-07-12 19:10:48,347 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-33fe1b8fcede4090a3b3e345bbc40189 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/testSplitTableRegionDeletedRowsDaughter=0e61b0eec6c8d265f5fb01657ce1d2a0-33fe1b8fcede4090a3b3e345bbc40189 2023-07-12 19:10:48,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/recovered.edits/111.seqid, newMaxSeqId=111, maxSeqId=106 2023-07-12 19:10:48,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:48,351 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e. 2023-07-12 19:10:48,351 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 22bf8485475c754fcd294e2642c4de0e: 2023-07-12 19:10:48,352 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:48,353 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=22bf8485475c754fcd294e2642c4de0e, regionState=CLOSED 2023-07-12 19:10:48,353 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689189048352"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189048352"}]},"ts":"1689189048352"} 2023-07-12 19:10:48,355 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-07-12 19:10:48,355 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 22bf8485475c754fcd294e2642c4de0e, server=jenkins-hbase20.apache.org,38353,1689189020667 in 223 msec 2023-07-12 19:10:48,357 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-07-12 19:10:48,357 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=22bf8485475c754fcd294e2642c4de0e, UNASSIGN in 226 msec 2023-07-12 19:10:48,357 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189048357"}]},"ts":"1689189048357"} 2023-07-12 19:10:48,358 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-07-12 19:10:48,359 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-07-12 19:10:48,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 238 msec 2023-07-12 19:10:48,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-12 19:10:48,431 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-07-12 19:10:48,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:48,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:48,438 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:48,440 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:48,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-07-12 19:10:48,448 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,448 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:48,448 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742 2023-07-12 19:10:48,450 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/recovered.edits] 2023-07-12 19:10:48,450 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/recovered.edits] 2023-07-12 19:10:48,450 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/recovered.edits] 2023-07-12 19:10:48,452 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/d56d65f62cb04298ae17e3520243bbc2 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf1/d56d65f62cb04298ae17e3520243bbc2 2023-07-12 19:10:48,453 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/86a85ae2d0e144debe53d815234947bb to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf1/86a85ae2d0e144debe53d815234947bb 2023-07-12 19:10:48,453 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/090cc5071bb041f69ad5258a25e1429d to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/090cc5071bb041f69ad5258a25e1429d 2023-07-12 19:10:48,454 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/45695b5932ee4511a54c041a405619f0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/45695b5932ee4511a54c041a405619f0 2023-07-12 19:10:48,455 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/943105fc4a8c45d797d5d95f66c14a9f to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/cf2/943105fc4a8c45d797d5d95f66c14a9f 2023-07-12 19:10:48,455 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/6f1432cdb469471dab3732646420e30a to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/cf2/6f1432cdb469471dab3732646420e30a 2023-07-12 19:10:48,455 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/72ccb17830a04baabfa21b59aad14e21 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/72ccb17830a04baabfa21b59aad14e21 2023-07-12 19:10:48,457 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/76a1a1a0f0fe4ea49628e7aa53b01b0f 2023-07-12 19:10:48,458 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/7a4f6744ef3248e6932fb626a8d2c353 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/7a4f6744ef3248e6932fb626a8d2c353 2023-07-12 19:10:48,459 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/recovered.edits/111.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e/recovered.edits/111.seqid 2023-07-12 19:10:48,459 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/recovered.edits/113.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742/recovered.edits/113.seqid 2023-07-12 19:10:48,459 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:10:48,460 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6328ad5feb88f12be2101cce567df742 2023-07-12 19:10:48,460 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/87d6d04db20f474ea2897ff4cae0719a to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/87d6d04db20f474ea2897ff4cae0719a 2023-07-12 19:10:48,461 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/8e92af53c5c34829b73d137ed716a284 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/8e92af53c5c34829b73d137ed716a284 2023-07-12 19:10:48,462 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/9e0699c405904b008f79ed9bd24ed491 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/9e0699c405904b008f79ed9bd24ed491 2023-07-12 19:10:48,463 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/bf0f657e1a804817b5b8e9e2d75bf42d to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/bf0f657e1a804817b5b8e9e2d75bf42d 2023-07-12 19:10:48,464 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/dc1d1e60c2b94a1291973f788a370e95 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf1/dc1d1e60c2b94a1291973f788a370e95 2023-07-12 19:10:48,466 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/1b6e38155cce49ec9d7b8478b011d7f4 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/1b6e38155cce49ec9d7b8478b011d7f4 2023-07-12 19:10:48,467 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/236c2e5d04814c849ee02ba7488bd884 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/236c2e5d04814c849ee02ba7488bd884 2023-07-12 19:10:48,468 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/33fe1b8fcede4090a3b3e345bbc40189 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/33fe1b8fcede4090a3b3e345bbc40189 2023-07-12 19:10:48,469 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/525cd0743cad400192a410acf27e8a5e to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/525cd0743cad400192a410acf27e8a5e 2023-07-12 19:10:48,470 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/94277c2ac28b4a4a9d2f8a60045fcc25 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/94277c2ac28b4a4a9d2f8a60045fcc25 2023-07-12 19:10:48,471 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/9587e226eb2e4ea79e074d63b8088932 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/9587e226eb2e4ea79e074d63b8088932 2023-07-12 19:10:48,472 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/95984a785fa4432aaa46b62e0ad439b2 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/95984a785fa4432aaa46b62e0ad439b2 2023-07-12 19:10:48,473 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/a0c5a22c1efb4dc99a241faa5a9b3080 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/a0c5a22c1efb4dc99a241faa5a9b3080 2023-07-12 19:10:48,474 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/b5167621f55e4c98a7ac8af680e28581 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/b5167621f55e4c98a7ac8af680e28581 2023-07-12 19:10:48,475 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/d747715a45d847d59970893fa9599993 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/cf2/d747715a45d847d59970893fa9599993 2023-07-12 19:10:48,477 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/recovered.edits/106.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0/recovered.edits/106.seqid 2023-07-12 19:10:48,478 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0e61b0eec6c8d265f5fb01657ce1d2a0 2023-07-12 19:10:48,478 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-07-12 19:10:48,480 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:48,481 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-07-12 19:10:48,483 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-07-12 19:10:48,484 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:48,484 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-07-12 19:10:48,484 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189048484"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:48,484 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189048484"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:48,484 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189048484"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:48,486 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-12 19:10:48,486 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0e61b0eec6c8d265f5fb01657ce1d2a0, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1689189044710.0e61b0eec6c8d265f5fb01657ce1d2a0.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 6328ad5feb88f12be2101cce567df742, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1689189046956.6328ad5feb88f12be2101cce567df742.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 22bf8485475c754fcd294e2642c4de0e, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1689189046956.22bf8485475c754fcd294e2642c4de0e.', STARTKEY => '60', ENDKEY => ''}] 2023-07-12 19:10:48,486 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-07-12 19:10:48,486 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689189048486"}]},"ts":"9223372036854775807"} 2023-07-12 19:10:48,487 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-07-12 19:10:48,489 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:10:48,490 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 56 msec 2023-07-12 19:10:48,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-07-12 19:10:48,543 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-07-12 19:10:48,571 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=410 (was 406) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:35916 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:35900 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:46426 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:35888 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:47962 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:46438 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:46394 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:48146 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:46424 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:35944 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:46408 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:46446 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:35940 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:47912 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:60856 [Waiting for operation #30] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49452fc5-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_900119405_17 at /127.0.0.1:46434 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=876 (was 874) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=188 (was 188), ProcessCount=170 (was 170), AvailableMemoryMB=5781 (was 5846) 2023-07-12 19:10:48,587 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=410, OpenFileDescriptor=876, MaxFileDescriptor=60000, SystemLoadAverage=188, ProcessCount=170, AvailableMemoryMB=5780 2023-07-12 19:10:48,587 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:10:48,590 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 19:10:48,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 19:10:48,593 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:10:48,595 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 19:10:48,596 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 19:10:48,597 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:48,598 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f empty. 2023-07-12 19:10:48,598 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:48,598 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-07-12 19:10:48,610 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-07-12 19:10:48,611 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7675): creating {ENCODED => cdf8695334a5bdad2bdb8206cb54526f, NAME => 'testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp 2023-07-12 19:10:48,622 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:48,622 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1604): Closing cdf8695334a5bdad2bdb8206cb54526f, disabling compactions & flushes 2023-07-12 19:10:48,622 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:10:48,623 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:10:48,623 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. after waiting 0 ms 2023-07-12 19:10:48,623 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:10:48,623 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:10:48,623 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1558): Region close journal for cdf8695334a5bdad2bdb8206cb54526f: 2023-07-12 19:10:48,625 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 19:10:48,626 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689189048626"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189048626"}]},"ts":"1689189048626"} 2023-07-12 19:10:48,627 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 19:10:48,628 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 19:10:48,628 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189048628"}]},"ts":"1689189048628"} 2023-07-12 19:10:48,629 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-07-12 19:10:48,633 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 19:10:48,633 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 19:10:48,633 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 19:10:48,633 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 19:10:48,633 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 19:10:48,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=cdf8695334a5bdad2bdb8206cb54526f, ASSIGN}] 2023-07-12 19:10:48,635 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=cdf8695334a5bdad2bdb8206cb54526f, ASSIGN 2023-07-12 19:10:48,635 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=cdf8695334a5bdad2bdb8206cb54526f, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,33255,1689189020635; forceNewPlan=false, retain=false 2023-07-12 19:10:48,786 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 19:10:48,792 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=cdf8695334a5bdad2bdb8206cb54526f, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:48,792 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689189048792"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189048792"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189048792"}]},"ts":"1689189048792"} 2023-07-12 19:10:48,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure cdf8695334a5bdad2bdb8206cb54526f, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:10:48,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:10:48,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cdf8695334a5bdad2bdb8206cb54526f, NAME => 'testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.', STARTKEY => '', ENDKEY => ''} 2023-07-12 19:10:48,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:48,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:48,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:48,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:48,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:48,954 INFO [StoreOpener-cdf8695334a5bdad2bdb8206cb54526f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:48,956 DEBUG [StoreOpener-cdf8695334a5bdad2bdb8206cb54526f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f/cf1 2023-07-12 19:10:48,956 DEBUG [StoreOpener-cdf8695334a5bdad2bdb8206cb54526f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f/cf1 2023-07-12 19:10:48,957 INFO [StoreOpener-cdf8695334a5bdad2bdb8206cb54526f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cdf8695334a5bdad2bdb8206cb54526f columnFamilyName cf1 2023-07-12 19:10:48,958 INFO [StoreOpener-cdf8695334a5bdad2bdb8206cb54526f-1] regionserver.HStore(310): Store=cdf8695334a5bdad2bdb8206cb54526f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:48,959 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:48,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:48,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:48,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 19:10:48,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened cdf8695334a5bdad2bdb8206cb54526f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71949453, jitterRate=0.07213039696216583}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 19:10:48,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for cdf8695334a5bdad2bdb8206cb54526f: 2023-07-12 19:10:48,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f., pid=87, masterSystemTime=1689189048948 2023-07-12 19:10:48,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:10:48,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:10:48,970 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=cdf8695334a5bdad2bdb8206cb54526f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:48,970 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689189048969"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189048969"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189048969"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189048969"}]},"ts":"1689189048969"} 2023-07-12 19:10:48,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-07-12 19:10:48,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure cdf8695334a5bdad2bdb8206cb54526f, server=jenkins-hbase20.apache.org,33255,1689189020635 in 175 msec 2023-07-12 19:10:48,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-07-12 19:10:48,976 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=cdf8695334a5bdad2bdb8206cb54526f, ASSIGN in 340 msec 2023-07-12 19:10:48,976 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 19:10:48,976 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189048976"}]},"ts":"1689189048976"} 2023-07-12 19:10:48,978 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-07-12 19:10:48,979 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 19:10:48,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 387 msec 2023-07-12 19:10:49,102 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$12(2454): Client=jenkins//148.251.75.209 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-12 19:10:49,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:10:49,121 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-07-12 19:10:49,122 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-07-12 19:10:49,123 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-07-12 19:10:49,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-07-12 19:10:49,136 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=cdf8695334a5bdad2bdb8206cb54526f, REOPEN/MOVE}] 2023-07-12 19:10:49,138 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=cdf8695334a5bdad2bdb8206cb54526f, REOPEN/MOVE 2023-07-12 19:10:49,138 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=cdf8695334a5bdad2bdb8206cb54526f, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:49,138 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689189049138"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189049138"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189049138"}]},"ts":"1689189049138"} 2023-07-12 19:10:49,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure cdf8695334a5bdad2bdb8206cb54526f, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:10:49,293 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:49,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing cdf8695334a5bdad2bdb8206cb54526f, disabling compactions & flushes 2023-07-12 19:10:49,295 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:10:49,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:10:49,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. after waiting 0 ms 2023-07-12 19:10:49,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:10:49,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-12 19:10:49,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:10:49,304 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:10:49,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for cdf8695334a5bdad2bdb8206cb54526f: 2023-07-12 19:10:49,305 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(3510): Not adding moved region record: cdf8695334a5bdad2bdb8206cb54526f to self. 2023-07-12 19:10:49,307 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:49,307 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=cdf8695334a5bdad2bdb8206cb54526f, regionState=CLOSED 2023-07-12 19:10:49,308 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689189049307"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189049307"}]},"ts":"1689189049307"} 2023-07-12 19:10:49,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-07-12 19:10:49,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure cdf8695334a5bdad2bdb8206cb54526f, server=jenkins-hbase20.apache.org,33255,1689189020635 in 169 msec 2023-07-12 19:10:49,312 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=cdf8695334a5bdad2bdb8206cb54526f, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase20.apache.org,33255,1689189020635; forceNewPlan=false, retain=true 2023-07-12 19:10:49,462 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 19:10:49,463 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=cdf8695334a5bdad2bdb8206cb54526f, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:49,463 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689189049463"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189049463"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189049463"}]},"ts":"1689189049463"} 2023-07-12 19:10:49,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure cdf8695334a5bdad2bdb8206cb54526f, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:10:49,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:10:49,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cdf8695334a5bdad2bdb8206cb54526f, NAME => 'testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.', STARTKEY => '', ENDKEY => ''} 2023-07-12 19:10:49,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:49,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:49,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:49,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:49,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:49,628 INFO [StoreOpener-cdf8695334a5bdad2bdb8206cb54526f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:49,630 DEBUG [StoreOpener-cdf8695334a5bdad2bdb8206cb54526f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f/cf1 2023-07-12 19:10:49,630 DEBUG [StoreOpener-cdf8695334a5bdad2bdb8206cb54526f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f/cf1 2023-07-12 19:10:49,631 INFO [StoreOpener-cdf8695334a5bdad2bdb8206cb54526f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cdf8695334a5bdad2bdb8206cb54526f columnFamilyName cf1 2023-07-12 19:10:49,631 INFO [StoreOpener-cdf8695334a5bdad2bdb8206cb54526f-1] regionserver.HStore(310): Store=cdf8695334a5bdad2bdb8206cb54526f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:10:49,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:49,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:49,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:10:49,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened cdf8695334a5bdad2bdb8206cb54526f; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61312272, jitterRate=-0.08637595176696777}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 19:10:49,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for cdf8695334a5bdad2bdb8206cb54526f: 2023-07-12 19:10:49,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f., pid=92, masterSystemTime=1689189049616 2023-07-12 19:10:49,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:10:49,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:10:49,640 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=cdf8695334a5bdad2bdb8206cb54526f, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:10:49,640 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689189049640"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189049640"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189049640"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189049640"}]},"ts":"1689189049640"} 2023-07-12 19:10:49,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-07-12 19:10:49,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure cdf8695334a5bdad2bdb8206cb54526f, server=jenkins-hbase20.apache.org,33255,1689189020635 in 176 msec 2023-07-12 19:10:49,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-07-12 19:10:49,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=cdf8695334a5bdad2bdb8206cb54526f, REOPEN/MOVE in 506 msec 2023-07-12 19:10:49,646 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-07-12 19:10:49,646 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 520 msec 2023-07-12 19:10:49,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50, ASSIGN}] 2023-07-12 19:10:49,651 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50, ASSIGN 2023-07-12 19:10:49,652 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-12 19:10:49,802 DEBUG [jenkins-hbase20:43565] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 19:10:49,804 DEBUG [jenkins-hbase20:43565] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 19:10:49,804 DEBUG [jenkins-hbase20:43565] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 19:10:49,804 DEBUG [jenkins-hbase20:43565] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 19:10:49,804 DEBUG [jenkins-hbase20:43565] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-07-12 19:10:49,807 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=a73a7d48997f0931e435d6a022fdfb50, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:10:49,807 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689189049806"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1689189049806"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1689189049806"}]},"ts":"1689189049806"} 2023-07-12 19:10:49,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure a73a7d48997f0931e435d6a022fdfb50, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:10:49,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1689189048591_0001.a73a7d48997f0931e435d6a022fdfb50. 2023-07-12 19:10:49,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a73a7d48997f0931e435d6a022fdfb50, NAME => 'testRollbackForSplitTableRegionWithReplica,,1689189048591_0001.a73a7d48997f0931e435d6a022fdfb50.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-07-12 19:10:49,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:10:49,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica a73a7d48997f0931e435d6a022fdfb50 2023-07-12 19:10:49,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1689189048591_0001.a73a7d48997f0931e435d6a022fdfb50.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:10:49,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for a73a7d48997f0931e435d6a022fdfb50 2023-07-12 19:10:49,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for a73a7d48997f0931e435d6a022fdfb50 2023-07-12 19:10:49,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:50,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:50,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:50,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:50,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:50,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:50,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:50,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:50,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:50,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:50,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:51,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:51,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:51,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:51,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:51,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:51,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:51,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:51,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:51,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:51,909 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-07-12 19:10:51,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:52,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:52,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:52,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:52,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:52,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:52,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:52,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:52,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:52,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:52,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:53,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:53,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:53,267 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 19:10:53,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:53,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:53,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:53,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:53,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:53,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:53,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:53,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:54,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:54,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:54,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:54,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:54,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:54,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:54,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:54,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:54,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:54,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:55,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:55,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:55,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:55,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:55,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:55,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:55,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:55,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:55,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:55,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:56,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:56,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:56,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:56,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:56,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:56,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:56,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:56,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:56,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:56,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:57,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:57,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:57,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:57,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:57,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:57,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:57,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:57,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:57,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:58,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:58,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:58,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:58,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:58,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:58,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:58,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:58,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:58,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:58,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:59,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:59,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:59,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:59,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:59,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:59,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:59,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:59,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:59,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:10:59,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:00,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:00,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:00,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:00,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:00,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:00,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:00,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:00,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:00,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:00,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:01,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:01,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:01,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:01,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:01,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:01,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:01,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:01,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:01,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:01,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:02,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:02,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:02,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:02,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:02,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:02,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:02,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:02,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:02,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:02,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:03,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:03,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:03,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:03,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:03,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:03,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:03,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:03,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:03,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:03,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:04,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:04,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:04,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:04,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:04,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:04,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:04,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:04,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:04,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:04,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:05,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:05,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:05,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:05,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:05,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:05,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:05,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:05,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:05,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:05,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:06,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:06,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:06,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:06,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:06,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:06,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:06,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:06,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:06,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:06,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:07,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:07,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:07,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:07,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:07,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:07,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:07,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:07,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:07,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:07,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:08,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:08,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:08,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:08,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:08,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:08,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:08,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:08,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:08,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:08,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:09,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:09,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:09,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:09,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:09,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:09,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:09,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:09,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:09,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:09,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:10,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:10,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:10,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:10,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:10,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:10,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:10,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:10,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:10,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:10,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:11,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:11,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:11,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:11,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:11,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:11,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:11,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:11,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:11,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:11,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:12,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:12,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:12,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:12,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:12,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:12,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:12,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:12,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:12,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:12,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:13,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:13,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:13,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:13,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:13,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:13,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:13,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:13,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:13,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:13,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:14,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:14,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:14,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:14,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:14,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:14,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:14,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:14,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:14,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:14,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:15,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:15,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:15,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:15,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:15,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:15,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:15,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:15,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:15,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:15,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:16,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:16,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:16,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:16,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:16,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:16,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:16,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:16,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:16,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:16,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:17,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:17,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:17,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:17,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:17,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:17,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:17,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:17,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:17,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:17,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:18,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:18,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:18,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:18,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:18,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:18,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:18,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:18,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:18,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:18,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:19,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:19,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:19,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:19,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:19,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:19,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:19,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:19,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:19,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:19,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:20,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:20,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:20,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:20,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:20,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:20,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:20,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:20,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:20,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:20,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:21,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:21,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:21,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:21,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:21,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:21,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:21,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:21,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:21,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:21,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:22,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:22,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:22,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:22,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:22,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:22,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:22,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:22,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:22,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:22,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:23,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:23,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:23,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:23,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:23,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:23,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:23,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:23,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:23,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:23,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:24,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:24,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:24,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:24,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:24,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:24,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:24,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:24,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:24,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:24,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:25,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:25,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:25,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:25,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:25,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:25,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:25,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:25,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:25,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:25,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:26,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:26,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:26,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:26,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:26,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:26,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:26,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:26,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:26,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:26,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:27,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:27,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:27,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:27,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:27,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:27,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:27,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:27,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:27,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:28,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:28,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:28,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:28,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:28,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:28,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:28,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:28,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:28,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:28,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:29,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:29,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:29,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:29,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:29,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:29,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:29,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:29,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:29,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:29,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:30,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:30,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:30,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:30,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:30,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:30,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:30,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:30,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:30,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:30,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:31,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:31,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:31,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:31,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:31,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:31,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:31,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:31,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:31,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:31,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:32,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:32,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:32,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:32,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:32,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:32,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:32,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:32,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:32,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:32,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:33,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:33,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:33,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:33,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:33,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:33,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:33,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:33,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:33,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:33,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:34,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:34,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:34,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:34,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:34,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:34,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:34,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:34,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:34,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:34,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:35,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:35,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:35,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:35,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:35,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:35,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:35,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:35,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:35,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:35,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:36,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:36,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:36,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:36,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:36,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:36,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:36,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:36,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:36,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:36,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:37,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:37,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:37,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:37,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:37,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:37,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:37,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:37,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:37,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:37,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:38,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:38,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:38,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:38,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:38,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:38,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:38,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:38,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:38,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:38,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:39,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:39,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:39,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:39,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:39,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:39,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:39,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:39,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:39,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:39,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:40,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:40,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:40,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:40,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:40,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:40,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:40,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:40,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:40,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:40,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:41,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:41,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:41,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:41,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:41,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:41,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:41,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:41,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:41,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:41,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:42,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:42,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:42,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:42,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:42,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:42,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:42,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:42,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:42,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:42,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:43,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:43,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:43,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:43,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:43,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:43,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:43,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:43,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:43,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:43,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:44,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:44,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:44,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:44,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:44,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:44,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:44,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:44,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:44,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:44,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:45,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:45,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:45,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:45,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:45,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:45,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:45,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:45,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:45,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:45,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:46,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:46,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:46,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:46,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:46,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:46,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:46,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:46,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:46,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:46,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:47,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:47,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:47,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:47,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:47,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:47,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:47,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:47,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:47,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:47,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:48,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:48,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:48,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:48,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:48,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:48,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:48,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:48,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:48,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:48,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:49,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:49,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:49,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-12 19:11:49,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:49,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-12 19:11:49,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:49,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:49,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-12 19:11:49,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:49,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:49,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:49,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-12 19:11:49,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:49,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:50,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:50,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:50,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:50,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-12 19:11:50,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:50,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:50,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:50,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:50,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:50,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:50,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:51,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:51,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:51,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:51,339 DEBUG [Listener at localhost.localdomain/40405] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:51,341 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 2023-07-12 19:11:51,343 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 2023-07-12 19:11:51,343 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 2023-07-12 19:11:51,343 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 2023-07-12 19:11:51,347 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting cdf8695334a5bdad2bdb8206cb54526f, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,38353,1689189020667, table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:11:51,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,38353,1689189020667, table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 exec-time=11 msec 2023-07-12 19:11:51,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:51,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:51,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 19:11:51,681 INFO [StoreOpener-a73a7d48997f0931e435d6a022fdfb50-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a73a7d48997f0931e435d6a022fdfb50 2023-07-12 19:11:51,689 DEBUG [StoreOpener-a73a7d48997f0931e435d6a022fdfb50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f/cf1 2023-07-12 19:11:51,689 DEBUG [StoreOpener-a73a7d48997f0931e435d6a022fdfb50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f/cf1 2023-07-12 19:11:51,690 INFO [StoreOpener-a73a7d48997f0931e435d6a022fdfb50-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a73a7d48997f0931e435d6a022fdfb50 columnFamilyName cf1 2023-07-12 19:11:51,691 INFO [StoreOpener-a73a7d48997f0931e435d6a022fdfb50-1] regionserver.HStore(310): Store=a73a7d48997f0931e435d6a022fdfb50/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:11:51,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened a73a7d48997f0931e435d6a022fdfb50; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69986857, jitterRate=0.0428854376077652}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 19:11:51,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for a73a7d48997f0931e435d6a022fdfb50: 2023-07-12 19:11:51,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1689189048591_0001.a73a7d48997f0931e435d6a022fdfb50., pid=94, masterSystemTime=1689189049965 2023-07-12 19:11:51,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1689189048591_0001.a73a7d48997f0931e435d6a022fdfb50. 2023-07-12 19:11:51,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1689189048591_0001.a73a7d48997f0931e435d6a022fdfb50. 2023-07-12 19:11:51,696 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=a73a7d48997f0931e435d6a022fdfb50, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:51,696 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689189111696"},{"qualifier":"server_0001","vlen":32,"tag":[],"timestamp":"1689189111696"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1689189111696"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1689189111696"}]},"ts":"1689189111696"} 2023-07-12 19:11:51,699 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-07-12 19:11:51,699 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure a73a7d48997f0931e435d6a022fdfb50, server=jenkins-hbase20.apache.org,38353,1689189020667 in 1 mins, 1.888 sec 2023-07-12 19:11:51,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-07-12 19:11:51,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50, ASSIGN in 1 mins, 2.05 sec 2023-07-12 19:11:51,704 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.597 sec 2023-07-12 19:11:51,704 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1689189049099","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62605,"client":"148.251.75.209:41996","queuetimems":0,"class":"HMaster"} 2023-07-12 19:11:51,704 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 603, skipping over this response of 3 bytes 2023-07-12 19:11:53,593 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,38353,1689189020667, table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50 is currently in transition, give up 2023-07-12 19:11:53,595 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:11:53,600 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:53,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:53,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:53,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-07-12 19:11:53,606 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189113606"}]},"ts":"1689189113606"} 2023-07-12 19:11:53,608 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-07-12 19:11:53,610 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-07-12 19:11:53,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=cdf8695334a5bdad2bdb8206cb54526f, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50, UNASSIGN}] 2023-07-12 19:11:53,612 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50, UNASSIGN 2023-07-12 19:11:53,613 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=a73a7d48997f0931e435d6a022fdfb50, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:53,613 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689189113613"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1689189113613"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1689189113613"}]},"ts":"1689189113613"} 2023-07-12 19:11:53,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure a73a7d48997f0931e435d6a022fdfb50, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:11:53,617 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=cdf8695334a5bdad2bdb8206cb54526f, UNASSIGN 2023-07-12 19:11:53,617 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=cdf8695334a5bdad2bdb8206cb54526f, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:53,617 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689189113617"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189113617"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189113617"}]},"ts":"1689189113617"} 2023-07-12 19:11:53,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure cdf8695334a5bdad2bdb8206cb54526f, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:11:53,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-07-12 19:11:53,768 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close a73a7d48997f0931e435d6a022fdfb50 2023-07-12 19:11:53,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing a73a7d48997f0931e435d6a022fdfb50, disabling compactions & flushes 2023-07-12 19:11:53,770 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1689189048591_0001.a73a7d48997f0931e435d6a022fdfb50. 2023-07-12 19:11:53,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1689189048591_0001.a73a7d48997f0931e435d6a022fdfb50. 2023-07-12 19:11:53,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1689189048591_0001.a73a7d48997f0931e435d6a022fdfb50. after waiting 0 ms 2023-07-12 19:11:53,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1689189048591_0001.a73a7d48997f0931e435d6a022fdfb50. 2023-07-12 19:11:53,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:11:53,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:11:53,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1689189048591_0001.a73a7d48997f0931e435d6a022fdfb50. 2023-07-12 19:11:53,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing cdf8695334a5bdad2bdb8206cb54526f, disabling compactions & flushes 2023-07-12 19:11:53,775 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:11:53,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:11:53,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for a73a7d48997f0931e435d6a022fdfb50: 2023-07-12 19:11:53,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. after waiting 0 ms 2023-07-12 19:11:53,776 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:11:53,780 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed a73a7d48997f0931e435d6a022fdfb50 2023-07-12 19:11:53,782 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=a73a7d48997f0931e435d6a022fdfb50, regionState=CLOSED 2023-07-12 19:11:53,782 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689189113782"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1689189113782"}]},"ts":"1689189113782"} 2023-07-12 19:11:53,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-07-12 19:11:53,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:11:53,787 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f. 2023-07-12 19:11:53,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for cdf8695334a5bdad2bdb8206cb54526f: 2023-07-12 19:11:53,788 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-07-12 19:11:53,788 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure a73a7d48997f0931e435d6a022fdfb50, server=jenkins-hbase20.apache.org,38353,1689189020667 in 169 msec 2023-07-12 19:11:53,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:11:53,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50, UNASSIGN in 177 msec 2023-07-12 19:11:53,790 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=cdf8695334a5bdad2bdb8206cb54526f, regionState=CLOSED 2023-07-12 19:11:53,790 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689189113790"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189113790"}]},"ts":"1689189113790"} 2023-07-12 19:11:53,793 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-07-12 19:11:53,793 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure cdf8695334a5bdad2bdb8206cb54526f, server=jenkins-hbase20.apache.org,33255,1689189020635 in 173 msec 2023-07-12 19:11:53,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-07-12 19:11:53,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=cdf8695334a5bdad2bdb8206cb54526f, UNASSIGN in 182 msec 2023-07-12 19:11:53,795 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189113795"}]},"ts":"1689189113795"} 2023-07-12 19:11:53,796 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-07-12 19:11:53,797 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-07-12 19:11:53,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 197 msec 2023-07-12 19:11:53,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-07-12 19:11:53,911 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-07-12 19:11:53,913 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:53,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:53,919 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:53,921 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:53,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-07-12 19:11:53,927 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:11:53,929 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f/recovered.edits] 2023-07-12 19:11:53,936 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f/recovered.edits/7.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f/recovered.edits/7.seqid 2023-07-12 19:11:53,936 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/cdf8695334a5bdad2bdb8206cb54526f 2023-07-12 19:11:53,936 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-07-12 19:11:53,939 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:53,941 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-07-12 19:11:53,943 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-07-12 19:11:53,944 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:53,944 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-07-12 19:11:53,944 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189113944"}]},"ts":"9223372036854775807"} 2023-07-12 19:11:53,944 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1689189048591_0001.a73a7d48997f0931e435d6a022fdfb50.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189113944"}]},"ts":"9223372036854775807"} 2023-07-12 19:11:53,946 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-07-12 19:11:53,946 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => cdf8695334a5bdad2bdb8206cb54526f, NAME => 'testRollbackForSplitTableRegionWithReplica,,1689189048591.cdf8695334a5bdad2bdb8206cb54526f.', STARTKEY => '', ENDKEY => ''}, {ENCODED => a73a7d48997f0931e435d6a022fdfb50, NAME => 'testRollbackForSplitTableRegionWithReplica,,1689189048591_0001.a73a7d48997f0931e435d6a022fdfb50.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-07-12 19:11:53,946 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-07-12 19:11:53,946 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689189113946"}]},"ts":"9223372036854775807"} 2023-07-12 19:11:53,948 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-07-12 19:11:53,949 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:53,950 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 36 msec 2023-07-12 19:11:54,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-07-12 19:11:54,022 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-07-12 19:11:54,036 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=346 (was 410), OpenFileDescriptor=659 (was 876), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=91 (was 188), ProcessCount=170 (was 170), AvailableMemoryMB=5786 (was 5780) - AvailableMemoryMB LEAK? - 2023-07-12 19:11:54,048 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=347, OpenFileDescriptor=659, MaxFileDescriptor=60000, SystemLoadAverage=91, ProcessCount=170, AvailableMemoryMB=5785 2023-07-12 19:11:54,049 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:11:54,051 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 19:11:54,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 19:11:54,053 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:54,055 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 19:11:54,055 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 19:11:54,057 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997 2023-07-12 19:11:54,057 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997 empty. 2023-07-12 19:11:54,058 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997 2023-07-12 19:11:54,058 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-07-12 19:11:54,067 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-07-12 19:11:54,067 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3f190266df01c9fc9d6d0bb73dd6e997, NAME => 'testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp 2023-07-12 19:11:54,077 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:11:54,077 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1604): Closing 3f190266df01c9fc9d6d0bb73dd6e997, disabling compactions & flushes 2023-07-12 19:11:54,077 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:54,077 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:54,077 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. after waiting 0 ms 2023-07-12 19:11:54,077 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:54,077 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:54,077 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1558): Region close journal for 3f190266df01c9fc9d6d0bb73dd6e997: 2023-07-12 19:11:54,079 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 19:11:54,080 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1689189114080"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189114080"}]},"ts":"1689189114080"} 2023-07-12 19:11:54,081 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 19:11:54,081 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 19:11:54,082 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189114081"}]},"ts":"1689189114081"} 2023-07-12 19:11:54,082 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-07-12 19:11:54,084 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 19:11:54,085 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 19:11:54,085 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 19:11:54,085 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 19:11:54,085 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 19:11:54,086 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=3f190266df01c9fc9d6d0bb73dd6e997, ASSIGN}] 2023-07-12 19:11:54,087 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=3f190266df01c9fc9d6d0bb73dd6e997, ASSIGN 2023-07-12 19:11:54,087 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=3f190266df01c9fc9d6d0bb73dd6e997, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,45655,1689189020596; forceNewPlan=false, retain=false 2023-07-12 19:11:54,238 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 19:11:54,239 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=3f190266df01c9fc9d6d0bb73dd6e997, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,239 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1689189114239"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189114239"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189114239"}]},"ts":"1689189114239"} 2023-07-12 19:11:54,242 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 3f190266df01c9fc9d6d0bb73dd6e997, server=jenkins-hbase20.apache.org,45655,1689189020596}] 2023-07-12 19:11:54,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:54,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3f190266df01c9fc9d6d0bb73dd6e997, NAME => 'testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997.', STARTKEY => '', ENDKEY => ''} 2023-07-12 19:11:54,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:11:54,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 3f190266df01c9fc9d6d0bb73dd6e997 2023-07-12 19:11:54,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:11:54,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 3f190266df01c9fc9d6d0bb73dd6e997 2023-07-12 19:11:54,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 3f190266df01c9fc9d6d0bb73dd6e997 2023-07-12 19:11:54,409 INFO [StoreOpener-3f190266df01c9fc9d6d0bb73dd6e997-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3f190266df01c9fc9d6d0bb73dd6e997 2023-07-12 19:11:54,412 DEBUG [StoreOpener-3f190266df01c9fc9d6d0bb73dd6e997-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1 2023-07-12 19:11:54,412 DEBUG [StoreOpener-3f190266df01c9fc9d6d0bb73dd6e997-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1 2023-07-12 19:11:54,412 INFO [StoreOpener-3f190266df01c9fc9d6d0bb73dd6e997-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3f190266df01c9fc9d6d0bb73dd6e997 columnFamilyName cf1 2023-07-12 19:11:54,413 INFO [StoreOpener-3f190266df01c9fc9d6d0bb73dd6e997-1] regionserver.HStore(310): Store=3f190266df01c9fc9d6d0bb73dd6e997/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:11:54,413 INFO [StoreOpener-3f190266df01c9fc9d6d0bb73dd6e997-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3f190266df01c9fc9d6d0bb73dd6e997 2023-07-12 19:11:54,415 DEBUG [StoreOpener-3f190266df01c9fc9d6d0bb73dd6e997-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2 2023-07-12 19:11:54,415 DEBUG [StoreOpener-3f190266df01c9fc9d6d0bb73dd6e997-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2 2023-07-12 19:11:54,416 INFO [StoreOpener-3f190266df01c9fc9d6d0bb73dd6e997-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3f190266df01c9fc9d6d0bb73dd6e997 columnFamilyName cf2 2023-07-12 19:11:54,417 INFO [StoreOpener-3f190266df01c9fc9d6d0bb73dd6e997-1] regionserver.HStore(310): Store=3f190266df01c9fc9d6d0bb73dd6e997/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:11:54,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997 2023-07-12 19:11:54,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997 2023-07-12 19:11:54,422 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:11:54,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 3f190266df01c9fc9d6d0bb73dd6e997 2023-07-12 19:11:54,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 19:11:54,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 3f190266df01c9fc9d6d0bb73dd6e997; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59233533, jitterRate=-0.11735157668590546}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:11:54,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 3f190266df01c9fc9d6d0bb73dd6e997: 2023-07-12 19:11:54,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997., pid=104, masterSystemTime=1689189114395 2023-07-12 19:11:54,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:54,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:54,431 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=3f190266df01c9fc9d6d0bb73dd6e997, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,431 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1689189114431"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189114431"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189114431"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189114431"}]},"ts":"1689189114431"} 2023-07-12 19:11:54,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-07-12 19:11:54,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 3f190266df01c9fc9d6d0bb73dd6e997, server=jenkins-hbase20.apache.org,45655,1689189020596 in 192 msec 2023-07-12 19:11:54,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-07-12 19:11:54,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=3f190266df01c9fc9d6d0bb73dd6e997, ASSIGN in 351 msec 2023-07-12 19:11:54,439 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 19:11:54,439 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189114439"}]},"ts":"1689189114439"} 2023-07-12 19:11:54,440 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-07-12 19:11:54,442 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 19:11:54,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 390 msec 2023-07-12 19:11:54,556 DEBUG [Listener at localhost.localdomain/40405] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 19:11:54,561 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:50772, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 19:11:54,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:11:54,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-12 19:11:54,570 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-12 19:11:54,570 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:54,570 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-12 19:11:54,570 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-12 19:11:54,571 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,571 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,572 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,572 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,572 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,572 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,572 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,572 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:11:54,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,572 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:54,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,573 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:54,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:54,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:54,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,573 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:54,573 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:54,573 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:54,573 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:54,573 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:54,573 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:54,573 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:54,573 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:54,573 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:54,573 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:54,573 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:54,573 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:54,573 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:54,574 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:54,573 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:54,574 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:54,574 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:54,574 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:54,574 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:54,574 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:11:54,574 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:11:54,574 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. started... 2023-07-12 19:11:54,575 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3f190266df01c9fc9d6d0bb73dd6e997 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 19:11:54,575 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,575 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,575 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,576 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:54,576 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,576 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:54,584 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf1/7f1fa769fdf34d6089965571dc27c9eb 2023-07-12 19:11:54,603 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf2/4608b17a27a1400a90c225a7760871fb 2023-07-12 19:11:54,610 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf1/7f1fa769fdf34d6089965571dc27c9eb as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/7f1fa769fdf34d6089965571dc27c9eb 2023-07-12 19:11:54,616 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/7f1fa769fdf34d6089965571dc27c9eb, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:11:54,617 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf2/4608b17a27a1400a90c225a7760871fb as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/4608b17a27a1400a90c225a7760871fb 2023-07-12 19:11:54,622 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/4608b17a27a1400a90c225a7760871fb, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:11:54,623 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 3f190266df01c9fc9d6d0bb73dd6e997 in 48ms, sequenceid=6, compaction requested=false 2023-07-12 19:11:54,623 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-07-12 19:11:54,623 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3f190266df01c9fc9d6d0bb73dd6e997: 2023-07-12 19:11:54,623 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:54,623 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:11:54,623 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:11:54,623 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,623 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:54,623 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:54,624 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,624 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:54,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:54,625 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,625 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:54,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:54,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:54,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:54,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-12 19:11:54,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8e0595b[Count = 0] remaining members to acquire global barrier 2023-07-12 19:11:54,627 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-12 19:11:54,627 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,627 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,627 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,627 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,628 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:54,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,628 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,628 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:54,628 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:54,628 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:54,628 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:54,628 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:54,628 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:11:54,628 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:11:54,628 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:11:54,628 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:11:54,629 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,629 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:54,629 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:54,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:54,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:54,629 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:54,629 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:54,629 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:54,629 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:54,629 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:54,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:54,629 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:54,629 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:54,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:54,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:54,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,45655,1689189020596': 2023-07-12 19:11:54,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-12 19:11:54,632 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-12 19:11:54,632 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:11:54,632 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:11:54,632 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-12 19:11:54,632 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:11:54,632 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,632 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,632 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,632 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,633 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,632 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,633 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,633 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,633 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,634 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:54,635 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,635 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,635 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,637 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,637 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,637 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,637 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,637 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,637 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-12 19:11:54,637 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:11:54,637 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,637 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,637 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:54,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,637 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,637 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-12 19:11:54,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,637 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,637 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,637 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:11:54,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,637 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,638 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,638 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,638 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,638 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,638 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,638 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,738 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:11:54,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:11:54,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:11:54,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-12 19:11:54,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-12 19:11:54,759 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-12 19:11:54,759 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:54,759 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-12 19:11:54,759 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-12 19:11:54,759 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,759 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,761 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,761 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,761 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,761 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,762 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,762 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:11:54,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,762 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,762 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,762 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:54,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:54,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,763 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:54,763 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:54,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:54,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,763 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:54,763 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 19:11:54,763 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:54,763 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:54,763 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 19:11:54,763 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:54,763 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 19:11:54,763 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:54,763 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:54,763 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:54,763 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:54,763 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:54,763 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:54,763 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:54,763 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:54,763 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:11:54,764 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:54,763 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:54,764 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. started... 2023-07-12 19:11:54,764 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:11:54,763 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:54,764 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:54,764 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3f190266df01c9fc9d6d0bb73dd6e997 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:11:54,764 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:54,764 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:54,765 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,765 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,765 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,765 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:54,765 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,765 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:54,773 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf1/9b70792a7b764025836e9f0bac1f0cbb 2023-07-12 19:11:54,784 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf2/3e7949264028492b93bc76ace60cf5a2 2023-07-12 19:11:54,788 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf1/9b70792a7b764025836e9f0bac1f0cbb as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/9b70792a7b764025836e9f0bac1f0cbb 2023-07-12 19:11:54,791 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/9b70792a7b764025836e9f0bac1f0cbb, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:11:54,791 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf2/3e7949264028492b93bc76ace60cf5a2 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/3e7949264028492b93bc76ace60cf5a2 2023-07-12 19:11:54,795 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/3e7949264028492b93bc76ace60cf5a2, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:11:54,796 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3f190266df01c9fc9d6d0bb73dd6e997 in 32ms, sequenceid=19, compaction requested=false 2023-07-12 19:11:54,796 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3f190266df01c9fc9d6d0bb73dd6e997: 2023-07-12 19:11:54,796 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:54,796 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:11:54,796 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:11:54,796 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,796 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:54,796 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:54,799 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,799 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:54,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:54,800 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,800 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:54,801 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-12 19:11:54,801 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ba9a431[Count = 0] remaining members to acquire global barrier 2023-07-12 19:11:54,801 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-12 19:11:54,802 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,802 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,802 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,802 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,802 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:54,802 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,802 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:11:54,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,802 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:54,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,803 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:11:54,803 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:54,803 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:54,803 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:11:54,803 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:54,803 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:54,803 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:11:54,803 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,803 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:54,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:54,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:54,803 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:54,803 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:54,803 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:54,803 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:54,804 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:54,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:54,804 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:54,804 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:54,804 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:54,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:54,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:54,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,45655,1689189020596': 2023-07-12 19:11:54,806 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-12 19:11:54,806 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-12 19:11:54,806 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:11:54,806 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:11:54,806 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-12 19:11:54,806 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:11:54,807 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,807 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,807 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,807 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,807 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,807 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,807 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:54,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:54,807 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,807 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:54,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,808 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:54,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,809 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,809 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:54,809 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,811 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,811 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,811 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,811 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,811 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,811 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,811 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,811 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-12 19:11:54,811 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:11:54,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,811 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,811 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,811 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,812 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,812 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,812 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-12 19:11:54,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,812 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,812 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,812 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,812 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:11:54,812 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,812 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,812 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,912 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:11:54,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:11:54,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:11:54,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-12 19:11:54,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-12 19:11:54,935 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-12 19:11:54,935 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:54,935 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-12 19:11:54,935 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-12 19:11:54,935 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,935 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,936 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,936 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,936 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,936 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,937 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,937 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:11:54,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:54,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:54,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:54,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,937 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:54,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:54,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:54,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 19:11:54,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 19:11:54,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:54,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 19:11:54,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:54,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:54,938 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:54,938 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:54,938 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:54,938 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:54,938 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:54,938 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:54,938 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:54,938 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:54,938 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:54,938 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:54,938 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:11:54,938 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:11:54,938 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:54,938 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:54,938 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:54,938 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. started... 2023-07-12 19:11:54,938 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:54,939 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3f190266df01c9fc9d6d0bb73dd6e997 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:11:54,939 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,939 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,939 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,940 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:54,940 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,940 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:54,947 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf1/7e7ed8a65dbf4e3ca51d02843db5f499 2023-07-12 19:11:54,961 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf2/458b4209d1fe436199c681af24d1bd53 2023-07-12 19:11:54,967 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf1/7e7ed8a65dbf4e3ca51d02843db5f499 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/7e7ed8a65dbf4e3ca51d02843db5f499 2023-07-12 19:11:54,971 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/7e7ed8a65dbf4e3ca51d02843db5f499, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:11:54,972 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf2/458b4209d1fe436199c681af24d1bd53 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/458b4209d1fe436199c681af24d1bd53 2023-07-12 19:11:54,976 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/458b4209d1fe436199c681af24d1bd53, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:11:54,977 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3f190266df01c9fc9d6d0bb73dd6e997 in 38ms, sequenceid=32, compaction requested=true 2023-07-12 19:11:54,977 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3f190266df01c9fc9d6d0bb73dd6e997: 2023-07-12 19:11:54,977 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:54,977 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:11:54,977 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:11:54,977 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,977 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:54,977 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:54,979 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,979 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:54,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:54,979 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,979 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:54,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:54,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:54,981 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-12 19:11:54,981 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3eaec1c[Count = 0] remaining members to acquire global barrier 2023-07-12 19:11:54,981 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-12 19:11:54,981 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,982 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,982 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,982 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,982 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:54,982 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,982 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:11:54,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,982 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:54,982 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:11:54,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,982 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:54,982 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:54,982 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:54,982 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:54,982 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:11:54,982 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:11:54,983 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:54,983 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:54,983 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:54,983 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:54,983 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:54,983 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:54,983 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,983 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:54,983 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:54,984 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:54,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,45655,1689189020596': 2023-07-12 19:11:54,986 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-12 19:11:54,986 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-12 19:11:54,986 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:11:54,986 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:11:54,986 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-12 19:11:54,986 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:11:54,987 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,987 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,987 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:54,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,987 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,987 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,987 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:54,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,988 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,988 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,988 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,988 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:54,990 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:54,990 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,990 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,993 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-12 19:11:54,993 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,993 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-12 19:11:54,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:54,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:54,993 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:11:54,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,094 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:11:55,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:11:55,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:11:55,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-12 19:11:55,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-12 19:11:55,107 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-12 19:11:55,107 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:55,107 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-12 19:11:55,107 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-12 19:11:55,108 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,108 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,109 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,109 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,109 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,109 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,109 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,109 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:11:55,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,110 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:55,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,110 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,110 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:55,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:55,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,110 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:55,110 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 19:11:55,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:55,110 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 19:11:55,110 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:55,110 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:55,110 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:55,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,110 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:55,110 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:55,110 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:55,110 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:55,110 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:55,110 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:55,110 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:55,111 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 19:11:55,110 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:55,111 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:55,111 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:55,111 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:55,111 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:11:55,111 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:55,111 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:55,111 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. started... 2023-07-12 19:11:55,111 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:11:55,111 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:55,111 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:55,111 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3f190266df01c9fc9d6d0bb73dd6e997 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:11:55,111 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,112 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,112 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:55,112 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,112 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,112 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:55,118 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf1/1f3e1199652f4ba18295edf9eb22a1f9 2023-07-12 19:11:55,129 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf2/6d444cc0407341eca7fe982ffef43ea0 2023-07-12 19:11:55,133 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf1/1f3e1199652f4ba18295edf9eb22a1f9 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/1f3e1199652f4ba18295edf9eb22a1f9 2023-07-12 19:11:55,136 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/1f3e1199652f4ba18295edf9eb22a1f9, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:11:55,137 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf2/6d444cc0407341eca7fe982ffef43ea0 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/6d444cc0407341eca7fe982ffef43ea0 2023-07-12 19:11:55,141 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/6d444cc0407341eca7fe982ffef43ea0, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:11:55,142 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3f190266df01c9fc9d6d0bb73dd6e997 in 31ms, sequenceid=45, compaction requested=true 2023-07-12 19:11:55,142 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3f190266df01c9fc9d6d0bb73dd6e997: 2023-07-12 19:11:55,142 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:55,142 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:11:55,142 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:11:55,142 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,142 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:55,142 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:55,143 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:55,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:55,144 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:55,144 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,144 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:55,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:55,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:55,145 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-12 19:11:55,145 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6fb06258[Count = 0] remaining members to acquire global barrier 2023-07-12 19:11:55,145 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-12 19:11:55,146 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,146 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,146 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,146 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,146 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,146 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:55,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,146 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:55,146 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:11:55,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,146 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:11:55,147 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:55,147 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:55,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,147 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:11:55,147 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:55,147 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:55,147 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:11:55,147 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:55,147 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:55,147 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:55,148 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:55,148 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:55,148 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:55,148 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,148 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:55,148 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:55,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:55,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:55,148 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:55,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:55,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:55,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:55,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,45655,1689189020596': 2023-07-12 19:11:55,150 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-12 19:11:55,150 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-12 19:11:55,150 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:11:55,150 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:11:55,150 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-12 19:11:55,150 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:11:55,151 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,151 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,151 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:55,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,151 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,151 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,151 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:55,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,152 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,152 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,152 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,153 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:55,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,154 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,154 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,154 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,156 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,156 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,156 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,156 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,156 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-12 19:11:55,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:11:55,156 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,156 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:55,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,156 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,156 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,156 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,156 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,156 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,156 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,156 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,156 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,157 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,157 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,157 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,156 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-12 19:11:55,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,157 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:11:55,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,257 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:11:55,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:11:55,269 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:11:55,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-12 19:11:55,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-12 19:11:55,270 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-12 19:11:55,270 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:55,270 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-12 19:11:55,270 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-12 19:11:55,271 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,271 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,271 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,271 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,271 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,271 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,272 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,272 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:11:55,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,272 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:55,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,273 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:55,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:55,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,273 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 19:11:55,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:55,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,273 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:55,273 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:55,273 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:55,273 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:55,273 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:55,273 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:55,273 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 19:11:55,273 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:55,273 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 19:11:55,273 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:55,273 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:11:55,273 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:11:55,273 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:55,273 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:55,273 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. started... 2023-07-12 19:11:55,273 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:55,273 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:55,273 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:55,274 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:55,274 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:55,274 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3f190266df01c9fc9d6d0bb73dd6e997 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:11:55,274 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:55,274 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:55,274 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:55,274 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:55,274 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,274 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,275 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,275 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,275 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:55,275 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:55,282 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf1/58fc62b272be4067b9cec54b62b4cbd3 2023-07-12 19:11:55,296 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf2/9333175befd24aefb0c0cf2c402f9c97 2023-07-12 19:11:55,300 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf1/58fc62b272be4067b9cec54b62b4cbd3 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/58fc62b272be4067b9cec54b62b4cbd3 2023-07-12 19:11:55,303 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/58fc62b272be4067b9cec54b62b4cbd3, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:11:55,304 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf2/9333175befd24aefb0c0cf2c402f9c97 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/9333175befd24aefb0c0cf2c402f9c97 2023-07-12 19:11:55,307 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/9333175befd24aefb0c0cf2c402f9c97, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:11:55,308 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3f190266df01c9fc9d6d0bb73dd6e997 in 34ms, sequenceid=58, compaction requested=true 2023-07-12 19:11:55,308 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3f190266df01c9fc9d6d0bb73dd6e997: 2023-07-12 19:11:55,308 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:55,308 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:11:55,308 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:11:55,308 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,308 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:55,308 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:55,310 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,310 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:55,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:55,310 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:55,310 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:55,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:55,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:55,312 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-12 19:11:55,312 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6510c474[Count = 0] remaining members to acquire global barrier 2023-07-12 19:11:55,312 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-12 19:11:55,312 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,313 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,313 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,313 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,313 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,313 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:11:55,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,313 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:55,313 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:55,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,313 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:11:55,313 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:55,313 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:55,313 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:11:55,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,313 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:55,313 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:55,314 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:11:55,314 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:55,314 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:55,314 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:55,314 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,314 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:55,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:55,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:55,314 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:55,314 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:55,315 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:55,315 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:55,315 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,45655,1689189020596': 2023-07-12 19:11:55,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-12 19:11:55,317 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-12 19:11:55,317 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:11:55,317 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:11:55,317 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-12 19:11:55,317 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:11:55,318 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,318 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,318 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,318 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,318 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,318 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,318 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,318 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:55,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,319 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:55,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,319 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:55,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,320 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:55,320 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,321 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,322 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,322 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,322 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,322 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,322 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,322 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,322 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,322 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,323 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,323 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-12 19:11:55,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:11:55,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,322 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,323 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:55,323 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,323 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-12 19:11:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,323 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:11:55,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,323 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,323 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,323 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,323 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,323 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,423 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:11:55,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:11:55,448 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:11:55,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-12 19:11:55,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-12 19:11:55,450 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-12 19:11:55,450 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:55,450 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-12 19:11:55,450 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-12 19:11:55,451 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,451 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,452 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,452 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,452 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,452 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,453 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,453 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:11:55,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:55,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,454 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:55,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:55,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:55,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,455 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 19:11:55,455 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:55,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:55,455 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 19:11:55,455 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:55,455 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 19:11:55,455 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:55,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:55,455 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 19:11:55,455 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:55,456 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:55,455 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:55,456 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:55,456 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:55,456 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:55,456 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:55,456 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:55,456 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 19:11:55,456 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 19:11:55,456 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:55,456 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 19:11:55,456 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:11:55,456 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:11:55,456 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:55,457 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. started... 2023-07-12 19:11:55,457 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3f190266df01c9fc9d6d0bb73dd6e997 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:11:55,457 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,457 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,458 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,458 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:55,458 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,458 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:55,469 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf1/0e0148595b9240398ff41074be0bbaed 2023-07-12 19:11:55,485 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf2/586bb1e2d6ad4e5d9b26a5d5b9f70e95 2023-07-12 19:11:55,490 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf1/0e0148595b9240398ff41074be0bbaed as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/0e0148595b9240398ff41074be0bbaed 2023-07-12 19:11:55,497 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/0e0148595b9240398ff41074be0bbaed, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:11:55,498 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf2/586bb1e2d6ad4e5d9b26a5d5b9f70e95 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/586bb1e2d6ad4e5d9b26a5d5b9f70e95 2023-07-12 19:11:55,506 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/586bb1e2d6ad4e5d9b26a5d5b9f70e95, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:11:55,507 INFO [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3f190266df01c9fc9d6d0bb73dd6e997 in 50ms, sequenceid=71, compaction requested=true 2023-07-12 19:11:55,507 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3f190266df01c9fc9d6d0bb73dd6e997: 2023-07-12 19:11:55,507 DEBUG [rs(jenkins-hbase20.apache.org,45655,1689189020596)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:55,507 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:11:55,507 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:11:55,507 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,507 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 19:11:55,508 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 19:11:55,509 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,509 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:55,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:55,510 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,510 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:55,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:55,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:55,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:55,513 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-12 19:11:55,513 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@433e7ccb[Count = 0] remaining members to acquire global barrier 2023-07-12 19:11:55,513 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-12 19:11:55,514 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,514 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,514 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,514 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,515 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,515 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:55,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,515 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:55,515 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:55,515 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 19:11:55,515 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:55,515 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:11:55,515 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 19:11:55,515 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:11:55,515 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:11:55,515 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:11:55,517 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:55,517 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,517 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:55,517 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:55,517 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:55,517 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:55,517 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 19:11:55,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:55,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:55,517 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:55,517 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:55,517 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 19:11:55,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:55,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:55,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:55,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,45655,1689189020596': 2023-07-12 19:11:55,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-12 19:11:55,524 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-12 19:11:55,524 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:11:55,524 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:11:55,524 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-12 19:11:55,524 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:11:55,533 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,533 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:55,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:55,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,533 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,533 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,533 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,534 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,534 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:55,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,534 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:55,536 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,537 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:55,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 19:11:55,540 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,541 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,541 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,544 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,544 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,544 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,544 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,544 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-12 19:11:55,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:11:55,544 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,544 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:55,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,544 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,545 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,545 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-12 19:11:55,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,545 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:11:55,545 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 19:11:55,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,545 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:55,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,545 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,545 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,545 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:55,545 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 19:11:55,546 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 19:11:55,550 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:55,550 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:55,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:55,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:55,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:55,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:55,645 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:11:55,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:11:55,657 INFO [Listener at localhost.localdomain/40405] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,657 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45655] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/9b70792a7b764025836e9f0bac1f0cbb because midkey is the same as first or last row 2023-07-12 19:11:55,658 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45655] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/3e7949264028492b93bc76ace60cf5a2 because midkey is the same as first or last row 2023-07-12 19:11:55,658 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45655] regionserver.HRegion(2745): Flushing 3f190266df01c9fc9d6d0bb73dd6e997 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-12 19:11:55,669 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45655] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf1/b6fb2db19f4d4270b570a002e901348f 2023-07-12 19:11:55,682 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45655] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf2/b2dca9fbe5e94850bd383bb5b74d5ad9 2023-07-12 19:11:55,686 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45655] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf1/b6fb2db19f4d4270b570a002e901348f as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/b6fb2db19f4d4270b570a002e901348f 2023-07-12 19:11:55,689 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45655] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/b6fb2db19f4d4270b570a002e901348f, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 19:11:55,690 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45655] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/.tmp/cf2/b2dca9fbe5e94850bd383bb5b74d5ad9 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/b2dca9fbe5e94850bd383bb5b74d5ad9 2023-07-12 19:11:55,694 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45655] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/b2dca9fbe5e94850bd383bb5b74d5ad9, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 19:11:55,695 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45655] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3f190266df01c9fc9d6d0bb73dd6e997 in 37ms, sequenceid=82, compaction requested=true 2023-07-12 19:11:55,696 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45655] regionserver.HRegion(2446): Flush status journal for 3f190266df01c9fc9d6d0bb73dd6e997: 2023-07-12 19:11:55,696 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45655] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/9b70792a7b764025836e9f0bac1f0cbb because midkey is the same as first or last row 2023-07-12 19:11:55,696 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45655] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/3e7949264028492b93bc76ace60cf5a2 because midkey is the same as first or last row 2023-07-12 19:11:55,696 DEBUG [Listener at localhost.localdomain/40405] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,697 DEBUG [Listener at localhost.localdomain/40405] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-07-12 19:11:55,697 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:11:55,699 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-07-12 19:11:55,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testInvalidSplitKey 2023-07-12 19:11:55,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:55,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-07-12 19:11:55,703 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189115703"}]},"ts":"1689189115703"} 2023-07-12 19:11:55,704 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-07-12 19:11:55,705 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-07-12 19:11:55,706 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=3f190266df01c9fc9d6d0bb73dd6e997, UNASSIGN}] 2023-07-12 19:11:55,707 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=3f190266df01c9fc9d6d0bb73dd6e997, UNASSIGN 2023-07-12 19:11:55,707 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=3f190266df01c9fc9d6d0bb73dd6e997, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:55,707 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1689189115707"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189115707"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189115707"}]},"ts":"1689189115707"} 2023-07-12 19:11:55,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 3f190266df01c9fc9d6d0bb73dd6e997, server=jenkins-hbase20.apache.org,45655,1689189020596}] 2023-07-12 19:11:55,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-07-12 19:11:55,862 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 3f190266df01c9fc9d6d0bb73dd6e997 2023-07-12 19:11:55,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 3f190266df01c9fc9d6d0bb73dd6e997, disabling compactions & flushes 2023-07-12 19:11:55,863 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:55,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:55,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. after waiting 0 ms 2023-07-12 19:11:55,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:55,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-12 19:11:55,869 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:11:55,869 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997. 2023-07-12 19:11:55,869 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 3f190266df01c9fc9d6d0bb73dd6e997: 2023-07-12 19:11:55,871 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 3f190266df01c9fc9d6d0bb73dd6e997 2023-07-12 19:11:55,871 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=3f190266df01c9fc9d6d0bb73dd6e997, regionState=CLOSED 2023-07-12 19:11:55,871 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1689189115871"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189115871"}]},"ts":"1689189115871"} 2023-07-12 19:11:55,875 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-07-12 19:11:55,875 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 3f190266df01c9fc9d6d0bb73dd6e997, server=jenkins-hbase20.apache.org,45655,1689189020596 in 164 msec 2023-07-12 19:11:55,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-07-12 19:11:55,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=3f190266df01c9fc9d6d0bb73dd6e997, UNASSIGN in 170 msec 2023-07-12 19:11:55,879 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189115879"}]},"ts":"1689189115879"} 2023-07-12 19:11:55,880 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-07-12 19:11:55,881 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-07-12 19:11:55,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 182 msec 2023-07-12 19:11:56,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-07-12 19:11:56,007 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-07-12 19:11:56,009 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testInvalidSplitKey 2023-07-12 19:11:56,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:56,013 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:56,014 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:56,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-07-12 19:11:56,020 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997 2023-07-12 19:11:56,024 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/recovered.edits] 2023-07-12 19:11:56,033 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/0e0148595b9240398ff41074be0bbaed to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/0e0148595b9240398ff41074be0bbaed 2023-07-12 19:11:56,036 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/1f3e1199652f4ba18295edf9eb22a1f9 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/1f3e1199652f4ba18295edf9eb22a1f9 2023-07-12 19:11:56,039 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/58fc62b272be4067b9cec54b62b4cbd3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/58fc62b272be4067b9cec54b62b4cbd3 2023-07-12 19:11:56,041 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/7e7ed8a65dbf4e3ca51d02843db5f499 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/7e7ed8a65dbf4e3ca51d02843db5f499 2023-07-12 19:11:56,044 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/7f1fa769fdf34d6089965571dc27c9eb to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/7f1fa769fdf34d6089965571dc27c9eb 2023-07-12 19:11:56,047 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/9b70792a7b764025836e9f0bac1f0cbb to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/9b70792a7b764025836e9f0bac1f0cbb 2023-07-12 19:11:56,049 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/b6fb2db19f4d4270b570a002e901348f to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf1/b6fb2db19f4d4270b570a002e901348f 2023-07-12 19:11:56,055 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/3e7949264028492b93bc76ace60cf5a2 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/3e7949264028492b93bc76ace60cf5a2 2023-07-12 19:11:56,057 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/458b4209d1fe436199c681af24d1bd53 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/458b4209d1fe436199c681af24d1bd53 2023-07-12 19:11:56,058 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/4608b17a27a1400a90c225a7760871fb to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/4608b17a27a1400a90c225a7760871fb 2023-07-12 19:11:56,060 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/586bb1e2d6ad4e5d9b26a5d5b9f70e95 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/586bb1e2d6ad4e5d9b26a5d5b9f70e95 2023-07-12 19:11:56,061 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/6d444cc0407341eca7fe982ffef43ea0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/6d444cc0407341eca7fe982ffef43ea0 2023-07-12 19:11:56,062 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/9333175befd24aefb0c0cf2c402f9c97 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/9333175befd24aefb0c0cf2c402f9c97 2023-07-12 19:11:56,063 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/b2dca9fbe5e94850bd383bb5b74d5ad9 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/cf2/b2dca9fbe5e94850bd383bb5b74d5ad9 2023-07-12 19:11:56,065 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/recovered.edits/85.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997/recovered.edits/85.seqid 2023-07-12 19:11:56,065 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testInvalidSplitKey/3f190266df01c9fc9d6d0bb73dd6e997 2023-07-12 19:11:56,065 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-07-12 19:11:56,067 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:56,068 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-07-12 19:11:56,071 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-07-12 19:11:56,072 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:56,072 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-07-12 19:11:56,072 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189116072"}]},"ts":"9223372036854775807"} 2023-07-12 19:11:56,074 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-12 19:11:56,074 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3f190266df01c9fc9d6d0bb73dd6e997, NAME => 'testInvalidSplitKey,,1689189114052.3f190266df01c9fc9d6d0bb73dd6e997.', STARTKEY => '', ENDKEY => ''}] 2023-07-12 19:11:56,074 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-07-12 19:11:56,074 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689189116074"}]},"ts":"9223372036854775807"} 2023-07-12 19:11:56,075 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-07-12 19:11:56,077 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:56,078 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 68 msec 2023-07-12 19:11:56,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-07-12 19:11:56,117 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-07-12 19:11:56,143 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=369 (was 347) - Thread LEAK? -, OpenFileDescriptor=745 (was 659) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=92 (was 91) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=5783 (was 5785) 2023-07-12 19:11:56,154 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=369, OpenFileDescriptor=745, MaxFileDescriptor=60000, SystemLoadAverage=92, ProcessCount=170, AvailableMemoryMB=5782 2023-07-12 19:11:56,155 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:11:56,157 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 19:11:56,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 19:11:56,159 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:11:56,160 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 19:11:56,161 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 19:11:56,163 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:56,163 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3 empty. 2023-07-12 19:11:56,164 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:56,164 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-07-12 19:11:56,176 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-07-12 19:11:56,177 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 68e5e51c22aa80fd52c4d2a0652112a3, NAME => 'testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp 2023-07-12 19:11:56,184 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:11:56,184 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 68e5e51c22aa80fd52c4d2a0652112a3, disabling compactions & flushes 2023-07-12 19:11:56,185 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:56,185 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:56,185 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. after waiting 0 ms 2023-07-12 19:11:56,185 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:56,185 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:56,185 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 68e5e51c22aa80fd52c4d2a0652112a3: 2023-07-12 19:11:56,187 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 19:11:56,187 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189116187"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189116187"}]},"ts":"1689189116187"} 2023-07-12 19:11:56,188 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 19:11:56,189 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 19:11:56,189 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189116189"}]},"ts":"1689189116189"} 2023-07-12 19:11:56,190 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-07-12 19:11:56,193 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 19:11:56,194 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 19:11:56,194 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 19:11:56,194 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 19:11:56,194 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 19:11:56,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, ASSIGN}] 2023-07-12 19:11:56,196 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, ASSIGN 2023-07-12 19:11:56,197 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,33255,1689189020635; forceNewPlan=false, retain=false 2023-07-12 19:11:56,347 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 19:11:56,350 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=68e5e51c22aa80fd52c4d2a0652112a3, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,351 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189116350"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189116350"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189116350"}]},"ts":"1689189116350"} 2023-07-12 19:11:56,355 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:11:56,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:56,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 68e5e51c22aa80fd52c4d2a0652112a3, NAME => 'testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3.', STARTKEY => '', ENDKEY => ''} 2023-07-12 19:11:56,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:11:56,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:56,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:11:56,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:56,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:56,520 INFO [StoreOpener-68e5e51c22aa80fd52c4d2a0652112a3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:56,522 DEBUG [StoreOpener-68e5e51c22aa80fd52c4d2a0652112a3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1 2023-07-12 19:11:56,522 DEBUG [StoreOpener-68e5e51c22aa80fd52c4d2a0652112a3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1 2023-07-12 19:11:56,522 INFO [StoreOpener-68e5e51c22aa80fd52c4d2a0652112a3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 68e5e51c22aa80fd52c4d2a0652112a3 columnFamilyName cf1 2023-07-12 19:11:56,523 INFO [StoreOpener-68e5e51c22aa80fd52c4d2a0652112a3-1] regionserver.HStore(310): Store=68e5e51c22aa80fd52c4d2a0652112a3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:11:56,523 INFO [StoreOpener-68e5e51c22aa80fd52c4d2a0652112a3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:56,525 DEBUG [StoreOpener-68e5e51c22aa80fd52c4d2a0652112a3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2 2023-07-12 19:11:56,525 DEBUG [StoreOpener-68e5e51c22aa80fd52c4d2a0652112a3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2 2023-07-12 19:11:56,525 INFO [StoreOpener-68e5e51c22aa80fd52c4d2a0652112a3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 68e5e51c22aa80fd52c4d2a0652112a3 columnFamilyName cf2 2023-07-12 19:11:56,526 INFO [StoreOpener-68e5e51c22aa80fd52c4d2a0652112a3-1] regionserver.HStore(310): Store=68e5e51c22aa80fd52c4d2a0652112a3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:11:56,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:56,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:56,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:11:56,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:56,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 19:11:56,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 68e5e51c22aa80fd52c4d2a0652112a3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64987868, jitterRate=-0.031605303287506104}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:11:56,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 68e5e51c22aa80fd52c4d2a0652112a3: 2023-07-12 19:11:56,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3., pid=111, masterSystemTime=1689189116508 2023-07-12 19:11:56,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:56,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:56,539 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=68e5e51c22aa80fd52c4d2a0652112a3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,539 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189116539"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189116539"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189116539"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189116539"}]},"ts":"1689189116539"} 2023-07-12 19:11:56,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-07-12 19:11:56,544 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 in 187 msec 2023-07-12 19:11:56,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-07-12 19:11:56,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, ASSIGN in 350 msec 2023-07-12 19:11:56,548 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 19:11:56,548 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189116548"}]},"ts":"1689189116548"} 2023-07-12 19:11:56,549 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-07-12 19:11:56,551 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 19:11:56,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 393 msec 2023-07-12 19:11:56,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:11:56,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-12 19:11:56,674 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-12 19:11:56,674 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:56,674 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-12 19:11:56,674 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 19:11:56,675 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,675 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,676 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:56,676 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:56,676 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:56,676 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:56,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:56,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:56,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:56,676 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,676 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:11:56,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:56,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:56,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:56,677 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:56,677 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:56,677 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:56,677 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:56,677 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:56,678 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:56,678 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:56,677 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:56,678 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:56,678 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:56,678 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:56,678 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:56,678 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:56,678 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:56,678 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:11:56,678 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:11:56,678 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:56,678 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. started... 2023-07-12 19:11:56,680 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 68e5e51c22aa80fd52c4d2a0652112a3 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 19:11:56,680 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,680 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,681 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,681 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,681 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:56,681 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:56,689 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf1/f8077d8e89bc4261bb003b6ad0bda3b1 2023-07-12 19:11:56,700 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf2/c9a72d8500a24cc7876cef4a7572be85 2023-07-12 19:11:56,705 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf1/f8077d8e89bc4261bb003b6ad0bda3b1 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/f8077d8e89bc4261bb003b6ad0bda3b1 2023-07-12 19:11:56,710 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/f8077d8e89bc4261bb003b6ad0bda3b1, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:11:56,712 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf2/c9a72d8500a24cc7876cef4a7572be85 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/c9a72d8500a24cc7876cef4a7572be85 2023-07-12 19:11:56,717 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/c9a72d8500a24cc7876cef4a7572be85, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:11:56,718 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 68e5e51c22aa80fd52c4d2a0652112a3 in 38ms, sequenceid=6, compaction requested=false 2023-07-12 19:11:56,718 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-07-12 19:11:56,719 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 68e5e51c22aa80fd52c4d2a0652112a3: 2023-07-12 19:11:56,719 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:56,719 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:11:56,719 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:11:56,719 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,719 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:56,719 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:56,721 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,721 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:56,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:56,721 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:56,721 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:56,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:56,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:56,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:56,724 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-12 19:11:56,724 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1760f2bc[Count = 0] remaining members to acquire global barrier 2023-07-12 19:11:56,724 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 19:11:56,725 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,725 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,725 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,725 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,726 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:56,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,726 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,726 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:11:56,726 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:56,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,726 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:11:56,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,726 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:56,726 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:56,726 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:56,726 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:56,726 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:11:56,726 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:11:56,727 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:56,727 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:56,727 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:56,728 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,728 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:56,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:56,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:56,728 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:56,728 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:56,728 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:56,728 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:56,728 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:56,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:56,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:56,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:56,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:56,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:56,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:11:56,731 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 19:11:56,731 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-12 19:11:56,731 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:11:56,731 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:11:56,731 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-12 19:11:56,731 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:11:56,732 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,732 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,733 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:56,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,732 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,733 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,733 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:56,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:56,733 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:56,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:56,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:56,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:56,733 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:56,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:56,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:56,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,733 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:56,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:56,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:56,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,734 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:56,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:56,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,735 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:56,735 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:56,736 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,738 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:56,738 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:56,738 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:56,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:56,738 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:56,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:56,738 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,738 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:56,738 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,738 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-12 19:11:56,738 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:11:56,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:56,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:56,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:56,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:56,738 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:56,738 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:56,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:56,738 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 19:11:56,738 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,738 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:11:56,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:56,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:56,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:56,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:56,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:56,738 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,739 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,739 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,739 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,739 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,739 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,739 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,739 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,839 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:11:56,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:11:56,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:11:56,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 19:11:56,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-12 19:11:56,864 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-12 19:11:56,864 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:56,865 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-12 19:11:56,865 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 19:11:56,865 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,865 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,866 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,866 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:56,866 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:56,866 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:56,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:56,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:56,866 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,866 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:11:56,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:56,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:56,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:56,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:56,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:56,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:56,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,867 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:56,867 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:56,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:56,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,867 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:56,867 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 19:11:56,867 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:56,867 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:56,867 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 19:11:56,867 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:56,867 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:56,867 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:56,867 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:56,867 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:56,867 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:56,868 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:56,868 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:56,868 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:56,868 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:56,869 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,869 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,869 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,869 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:56,869 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,869 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:56,870 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 19:11:56,870 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:56,871 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:56,871 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:56,871 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:56,877 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:11:56,877 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:11:56,877 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:56,877 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. started... 2023-07-12 19:11:56,877 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 68e5e51c22aa80fd52c4d2a0652112a3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:11:56,886 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf1/81615d3e65754dcd84f6b452c189d7ff 2023-07-12 19:11:56,897 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf2/99cba98576fe41958f05e66f4613c320 2023-07-12 19:11:56,900 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf1/81615d3e65754dcd84f6b452c189d7ff as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/81615d3e65754dcd84f6b452c189d7ff 2023-07-12 19:11:56,904 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/81615d3e65754dcd84f6b452c189d7ff, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:11:56,904 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf2/99cba98576fe41958f05e66f4613c320 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/99cba98576fe41958f05e66f4613c320 2023-07-12 19:11:56,907 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/99cba98576fe41958f05e66f4613c320, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:11:56,908 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 68e5e51c22aa80fd52c4d2a0652112a3 in 31ms, sequenceid=19, compaction requested=false 2023-07-12 19:11:56,908 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 68e5e51c22aa80fd52c4d2a0652112a3: 2023-07-12 19:11:56,908 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:56,908 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:11:56,908 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:11:56,908 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,908 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:56,908 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:56,909 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,909 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:56,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:56,910 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,910 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:56,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:56,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:56,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:56,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:56,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-12 19:11:56,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b2ee0aa[Count = 0] remaining members to acquire global barrier 2023-07-12 19:11:56,912 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 19:11:56,912 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,912 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,912 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,912 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,913 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:56,913 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:56,913 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,913 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:11:56,913 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:56,913 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:11:56,913 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:56,913 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:56,913 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:56,913 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:11:56,913 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:11:56,914 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:56,914 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:56,914 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:56,914 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:56,914 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:56,914 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:56,914 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,914 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:56,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:56,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:56,914 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:56,914 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:56,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:56,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:56,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:56,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:56,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:56,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:11:56,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 19:11:56,917 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-12 19:11:56,917 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:11:56,917 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:11:56,917 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-12 19:11:56,917 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:11:56,917 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,917 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,917 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,918 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:56,917 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,917 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:56,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:56,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:56,918 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,918 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:56,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:56,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:56,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:56,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:56,918 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:56,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:56,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:56,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:56,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,919 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:56,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:56,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:56,920 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:56,920 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,920 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,922 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,922 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:56,922 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:56,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:56,922 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:56,922 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:56,922 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,922 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-12 19:11:56,922 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:11:56,922 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,922 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:56,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:56,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:56,922 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:56,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:56,922 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:56,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:56,922 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 19:11:56,922 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:56,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:56,923 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:11:56,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:56,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:56,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:56,923 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,923 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:56,923 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:56,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:56,923 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:56,923 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:56,923 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:56,923 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,023 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:11:57,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:11:57,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:11:57,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 19:11:57,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-12 19:11:57,050 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-12 19:11:57,050 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,050 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-12 19:11:57,050 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 19:11:57,051 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,051 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,052 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,052 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,052 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,052 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,053 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,053 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:11:57,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,054 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,054 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,054 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:57,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:57,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:57,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,054 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:57,054 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:57,054 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:57,055 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 19:11:57,055 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 19:11:57,055 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,055 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 19:11:57,055 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,055 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,055 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:57,055 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,055 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:57,055 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:57,055 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,055 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:57,055 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:11:57,056 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:11:57,055 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:57,056 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:57,056 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:57,056 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:57,055 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,056 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. started... 2023-07-12 19:11:57,056 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:57,056 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:57,056 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:57,056 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 68e5e51c22aa80fd52c4d2a0652112a3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:11:57,058 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,059 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,059 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,059 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:57,059 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,059 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:57,068 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf1/d0bcf72680864ebc9e117386fec224ee 2023-07-12 19:11:57,082 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf2/bae3f6c2bf974087be57a28f2b9b6524 2023-07-12 19:11:57,087 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf1/d0bcf72680864ebc9e117386fec224ee as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/d0bcf72680864ebc9e117386fec224ee 2023-07-12 19:11:57,092 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/d0bcf72680864ebc9e117386fec224ee, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:11:57,093 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf2/bae3f6c2bf974087be57a28f2b9b6524 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/bae3f6c2bf974087be57a28f2b9b6524 2023-07-12 19:11:57,096 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/bae3f6c2bf974087be57a28f2b9b6524, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:11:57,097 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 68e5e51c22aa80fd52c4d2a0652112a3 in 41ms, sequenceid=32, compaction requested=true 2023-07-12 19:11:57,097 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 68e5e51c22aa80fd52c4d2a0652112a3: 2023-07-12 19:11:57,097 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:57,097 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:11:57,097 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:11:57,097 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,097 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:57,097 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:57,098 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,098 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:57,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:57,098 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:57,098 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:57,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:57,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:57,100 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-12 19:11:57,100 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@187eb8df[Count = 0] remaining members to acquire global barrier 2023-07-12 19:11:57,100 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 19:11:57,100 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,101 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,101 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,101 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,101 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,102 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:11:57,101 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:57,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,102 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:57,102 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:11:57,101 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:57,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,102 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:57,102 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:57,102 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:57,102 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:11:57,102 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:11:57,103 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,103 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:57,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:57,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:57,103 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,103 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:57,103 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:57,103 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,103 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:57,104 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:57,104 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,104 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:57,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:57,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:57,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:57,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:11:57,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 19:11:57,108 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-12 19:11:57,109 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:11:57,109 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:11:57,109 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,109 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:11:57,116 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,116 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:57,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:57,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,117 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,116 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,116 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:57,117 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,118 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,117 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,118 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:57,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,119 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:57,120 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,120 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,120 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,122 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,122 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,122 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,122 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-12 19:11:57,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:11:57,122 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,122 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,122 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,123 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,123 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,123 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,123 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 19:11:57,123 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,123 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:11:57,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,123 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,123 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,123 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,123 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,123 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,126 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,126 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,223 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:11:57,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:11:57,244 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:11:57,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 19:11:57,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-12 19:11:57,246 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-12 19:11:57,246 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,246 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-12 19:11:57,246 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 19:11:57,246 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,247 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,248 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,248 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,248 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,248 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,248 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,248 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:11:57,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:57,249 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,249 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 19:11:57,249 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:57,250 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,249 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,250 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:57,250 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:57,250 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,250 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 19:11:57,250 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:57,250 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:57,250 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:57,250 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:57,250 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,250 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:57,250 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:57,250 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,250 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:57,250 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:11:57,250 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:11:57,250 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:57,250 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. started... 2023-07-12 19:11:57,250 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 68e5e51c22aa80fd52c4d2a0652112a3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:11:57,250 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,251 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,251 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,251 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:57,251 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,251 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:57,257 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf1/99c53f93030b456abb8c93f19343df11 2023-07-12 19:11:57,269 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf2/63323db36ea8495c8b004cfce8d8176d 2023-07-12 19:11:57,273 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf1/99c53f93030b456abb8c93f19343df11 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/99c53f93030b456abb8c93f19343df11 2023-07-12 19:11:57,278 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/99c53f93030b456abb8c93f19343df11, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:11:57,279 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf2/63323db36ea8495c8b004cfce8d8176d as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/63323db36ea8495c8b004cfce8d8176d 2023-07-12 19:11:57,284 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/63323db36ea8495c8b004cfce8d8176d, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:11:57,285 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 68e5e51c22aa80fd52c4d2a0652112a3 in 35ms, sequenceid=45, compaction requested=true 2023-07-12 19:11:57,285 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 68e5e51c22aa80fd52c4d2a0652112a3: 2023-07-12 19:11:57,285 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:57,285 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:11:57,285 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:11:57,285 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,285 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:57,285 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:57,287 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,287 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:57,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:57,287 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,287 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:57,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:57,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:57,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:57,291 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-12 19:11:57,291 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2816473f[Count = 0] remaining members to acquire global barrier 2023-07-12 19:11:57,291 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 19:11:57,291 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,291 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,291 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,291 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,292 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,292 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:57,292 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:57,292 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:57,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,292 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:11:57,292 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:57,292 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:11:57,292 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:11:57,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,292 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:57,292 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:57,292 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:11:57,292 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:57,292 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,292 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:57,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:57,293 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:57,293 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:57,293 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,293 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:57,293 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:57,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:57,293 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,293 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:57,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:57,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:57,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:11:57,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 19:11:57,295 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-12 19:11:57,295 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:11:57,295 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:11:57,295 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,295 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:11:57,296 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,296 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:57,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:57,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,296 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,296 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,296 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,296 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:57,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,297 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,296 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,297 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:57,297 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:57,298 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,299 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,299 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,300 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,300 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,300 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,300 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,300 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,300 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,301 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,301 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,301 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,301 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,301 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,301 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,301 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,301 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,301 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,301 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,301 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,301 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,301 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-12 19:11:57,301 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:11:57,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,302 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 19:11:57,302 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:11:57,402 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:11:57,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:11:57,426 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:11:57,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 19:11:57,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-12 19:11:57,428 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-12 19:11:57,428 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,428 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-12 19:11:57,428 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 19:11:57,429 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,429 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,430 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,430 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,430 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,430 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,430 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,430 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:11:57,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,431 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,431 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,431 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:57,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:57,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,431 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:57,431 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:57,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:57,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,431 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:57,432 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 19:11:57,432 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 19:11:57,432 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,432 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,432 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:57,432 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,432 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 19:11:57,432 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:57,432 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:57,432 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:57,432 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,432 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:57,432 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,432 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:57,432 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:57,432 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,432 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:57,432 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:57,432 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:57,432 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:11:57,432 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:11:57,432 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:57,432 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. started... 2023-07-12 19:11:57,433 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 68e5e51c22aa80fd52c4d2a0652112a3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:11:57,434 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,434 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,434 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,434 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:57,434 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,435 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:57,442 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf1/229a06affadc465bad0eddfa6f69d629 2023-07-12 19:11:57,456 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf2/108649e022dc4c9bb30deaf5bd890260 2023-07-12 19:11:57,463 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf1/229a06affadc465bad0eddfa6f69d629 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/229a06affadc465bad0eddfa6f69d629 2023-07-12 19:11:57,468 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/229a06affadc465bad0eddfa6f69d629, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:11:57,469 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf2/108649e022dc4c9bb30deaf5bd890260 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/108649e022dc4c9bb30deaf5bd890260 2023-07-12 19:11:57,473 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/108649e022dc4c9bb30deaf5bd890260, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:11:57,474 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 68e5e51c22aa80fd52c4d2a0652112a3 in 41ms, sequenceid=58, compaction requested=true 2023-07-12 19:11:57,474 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 68e5e51c22aa80fd52c4d2a0652112a3: 2023-07-12 19:11:57,474 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:57,474 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:11:57,474 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:11:57,474 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,474 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:57,474 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:57,475 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,475 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:57,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:57,476 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,476 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:57,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:57,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:57,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:57,478 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-12 19:11:57,478 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3873b593[Count = 0] remaining members to acquire global barrier 2023-07-12 19:11:57,478 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 19:11:57,478 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,479 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,479 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,479 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,480 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,480 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:11:57,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,480 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:57,480 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:57,480 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:11:57,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,480 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:57,480 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:57,480 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:11:57,480 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:57,480 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:57,480 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:11:57,481 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:57,481 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,481 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:57,481 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:57,481 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,481 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:57,482 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:57,482 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,482 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:57,482 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:57,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:57,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:57,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:57,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:57,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:11:57,487 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 19:11:57,487 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-12 19:11:57,487 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:11:57,487 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:11:57,487 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,487 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:11:57,500 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,500 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:57,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:57,500 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,500 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,501 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,500 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:57,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,501 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,501 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,502 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:57,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,503 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,506 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,507 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,507 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,511 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,511 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,511 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,511 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,511 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,511 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,511 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,512 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,511 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,511 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-12 19:11:57,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:11:57,512 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,513 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,513 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 19:11:57,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,513 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:11:57,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,513 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,513 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,513 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,513 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,513 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,514 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,613 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:11:57,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:11:57,628 ERROR [BP-1020377755-148.251.75.209-1689189016963 heartbeating to localhost.localdomain/127.0.0.1:43855] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1020377755-148.251.75.209-1689189016963 (Datanode Uuid 20bbb2fd-7c0b-4edb-836c-47234fa474c0) service to localhost.localdomain/127.0.0.1:43855 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-12 19:11:57,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:11:57,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 19:11:57,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-12 19:11:57,632 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-12 19:11:57,633 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,633 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-12 19:11:57,633 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 19:11:57,633 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,633 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,634 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,634 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,634 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,634 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,635 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,635 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:11:57,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,635 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,635 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,635 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:57,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:57,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:57,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:11:57,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:57,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 19:11:57,636 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 19:11:57,636 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,636 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:57,636 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,636 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 19:11:57,636 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:57,636 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:57,636 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 19:11:57,636 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:57,636 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,636 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,636 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:57,636 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,636 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 19:11:57,637 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:11:57,636 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:57,637 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 19:11:57,637 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:57,637 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:57,637 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:11:57,637 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:11:57,637 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:57,637 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. started... 2023-07-12 19:11:57,637 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 68e5e51c22aa80fd52c4d2a0652112a3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:11:57,637 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,638 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,638 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,638 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:57,638 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,638 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:57,646 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf1/2187000c9c5d477c998f64a30937a0a9 2023-07-12 19:11:57,657 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf2/9a19ab58e24f463086c2367aea7becd5 2023-07-12 19:11:57,664 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf1/2187000c9c5d477c998f64a30937a0a9 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/2187000c9c5d477c998f64a30937a0a9 2023-07-12 19:11:57,668 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/2187000c9c5d477c998f64a30937a0a9, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:11:57,669 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf2/9a19ab58e24f463086c2367aea7becd5 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/9a19ab58e24f463086c2367aea7becd5 2023-07-12 19:11:57,672 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/9a19ab58e24f463086c2367aea7becd5, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:11:57,673 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 68e5e51c22aa80fd52c4d2a0652112a3 in 36ms, sequenceid=71, compaction requested=true 2023-07-12 19:11:57,673 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 68e5e51c22aa80fd52c4d2a0652112a3: 2023-07-12 19:11:57,673 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:57,673 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:11:57,673 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:11:57,673 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,673 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 19:11:57,673 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 19:11:57,675 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,675 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:57,675 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,675 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:11:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:57,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:57,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-12 19:11:57,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34542987[Count = 0] remaining members to acquire global barrier 2023-07-12 19:11:57,677 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 19:11:57,677 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,677 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,677 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,677 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,677 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,678 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:57,678 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:57,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,678 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:57,678 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:57,678 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:11:57,678 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:11:57,678 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:11:57,678 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 19:11:57,678 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 19:11:57,678 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:11:57,678 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:57,678 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,678 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:57,680 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:57,680 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,680 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,680 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 19:11:57,680 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,680 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:57,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:57,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:57,680 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 19:11:57,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:57,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:11:57,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 19:11:57,684 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-12 19:11:57,684 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:11:57,684 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:11:57,684 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-12 19:11:57,684 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:11:57,691 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,691 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,691 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,691 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,691 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:11:57,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:11:57,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,691 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,692 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,691 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:11:57,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,692 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,692 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:11:57,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,694 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:11:57,694 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 19:11:57,694 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,700 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,700 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,700 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,700 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:11:57,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-12 19:11:57,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:11:57,700 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:11:57,700 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,700 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,700 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:11:57,700 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,700 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 19:11:57,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:11:57,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:11:57,700 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:11:57,700 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,701 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:11:57,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:11:57,701 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,701 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:57,701 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,701 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:11:57,701 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:11:57,701 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,701 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 19:11:57,801 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:11:57,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:11:58,071 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-07-12 19:11:58,073 DEBUG [Listener at localhost.localdomain/40405] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:58,075 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,077 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,077 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,077 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,078 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:11:58,078 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,078 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:11:58,078 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:11:58,326 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,326 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:11:58,327 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:11:58,327 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:11:58,327 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-07-12 19:11:58,328 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:11:58,328 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:11:58,328 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:11:58,329 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:11:58,330 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:11:58,335 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-07-12 19:11:58,336 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:11:58,337 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:11:58,338 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:11:58,338 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:11:58,339 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:11:58,339 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:11:58,339 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:11:58,339 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:11:58,340 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:11:58,340 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:11:58,340 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:11:58,340 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:11:58,340 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:11:58,341 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:11:58,341 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:11:58,341 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:11:58,341 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:11:58,341 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:11:58,341 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb 2023-07-12 19:11:58,342 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:11:58,342 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:11:58,342 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:11:58,342 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:11:58,342 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:11:58,342 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:11:58,343 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:58,343 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:58,343 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,38353,1689189020667, table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 2023-07-12 19:11:58,344 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:58,344 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:58,344 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:58,344 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:58,344 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:58,344 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:11:58,345 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,345 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:11:58,345 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,345 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,345 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,345 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 14 msec 2023-07-12 19:11:58,346 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:11:58,346 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:11:58,347 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189148347 2023-07-12 19:11:58,348 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:11:58,348 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,351 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:11:58,352 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=68e5e51c22aa80fd52c4d2a0652112a3, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:11:58,352 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:11:58,352 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189178352 2023-07-12 19:11:58,352 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189238352 2023-07-12 19:11:58,352 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 19:11:58,352 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:11:58,354 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:11:58,354 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:11:58,354 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:11:58,602 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,603 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:11:58,603 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:11:58,603 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:11:58,604 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:11:58,604 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:11:58,604 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:11:58,604 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:11:58,605 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:11:58,606 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:11:58,611 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-12 19:11:58,611 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:11:58,611 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:11:58,612 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:11:58,612 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:11:58,612 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:11:58,612 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:11:58,612 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:11:58,612 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:11:58,613 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:11:58,613 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:11:58,613 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:11:58,613 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:11:58,613 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:11:58,614 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:11:58,614 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:11:58,614 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:11:58,614 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:11:58,614 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:11:58,614 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb 2023-07-12 19:11:58,614 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:11:58,615 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:11:58,615 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:11:58,615 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:11:58,615 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:11:58,615 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:11:58,615 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:58,615 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:58,616 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,38353,1689189020667, table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 2023-07-12 19:11:58,616 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:58,616 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:58,616 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:58,616 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:58,616 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:58,616 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:11:58,617 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,617 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:11:58,617 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,617 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,617 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,617 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-12 19:11:58,618 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:11:58,618 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:11:58,619 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189148619 2023-07-12 19:11:58,620 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:11:58,620 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,622 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:11:58,623 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=68e5e51c22aa80fd52c4d2a0652112a3, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:11:58,623 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:11:58,623 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189178623 2023-07-12 19:11:58,623 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189238623 2023-07-12 19:11:58,623 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:11:58,623 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:11:58,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN}] 2023-07-12 19:11:58,625 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:11:58,625 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:11:58,625 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:11:58,873 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,873 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:11:58,873 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:11:58,873 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:11:58,874 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:11:58,874 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:11:58,874 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:11:58,874 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:11:58,874 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:11:58,875 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 19:11:58,877 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-12 19:11:58,877 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:11:58,878 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:11:58,878 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:11:58,878 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:11:58,878 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:11:58,879 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:11:58,879 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:11:58,879 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:11:58,880 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:11:58,880 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:11:58,880 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:11:58,880 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:11:58,881 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:11:58,881 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:11:58,881 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:11:58,882 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:11:58,882 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:11:58,882 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:11:58,882 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb 2023-07-12 19:11:58,882 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:11:58,883 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:11:58,883 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:11:58,883 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:11:58,884 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:11:58,884 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:11:58,884 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:58,884 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:58,885 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,38353,1689189020667, table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 2023-07-12 19:11:58,885 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:58,885 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:58,886 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:58,886 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:58,886 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:58,886 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:11:58,887 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,887 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:11:58,887 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,887 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,887 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,887 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-07-12 19:11:58,888 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:11:58,889 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:11:58,890 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189148890 2023-07-12 19:11:58,892 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:11:58,892 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:58,894 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:11:58,894 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=68e5e51c22aa80fd52c4d2a0652112a3, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:11:58,894 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:11:58,894 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189178894 2023-07-12 19:11:58,894 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189238894 2023-07-12 19:11:58,894 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 19:11:58,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN}] 2023-07-12 19:11:58,895 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:11:58,897 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN 2023-07-12 19:11:58,898 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=68e5e51c22aa80fd52c4d2a0652112a3, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:58,898 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189118898"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189118898"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189118898"}]},"ts":"1689189118898"} 2023-07-12 19:11:58,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:11:59,052 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 68e5e51c22aa80fd52c4d2a0652112a3, disabling compactions & flushes 2023-07-12 19:11:59,052 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:59,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:59,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. after waiting 0 ms 2023-07-12 19:11:59,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:59,053 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 68e5e51c22aa80fd52c4d2a0652112a3 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-12 19:11:59,068 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf1/bf2ff10742b848baa3e39acc2c14d5d3 2023-07-12 19:11:59,082 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf2/f7bf0c8258c844c6955eaf3283b45e59 2023-07-12 19:11:59,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf1/bf2ff10742b848baa3e39acc2c14d5d3 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/bf2ff10742b848baa3e39acc2c14d5d3 2023-07-12 19:11:59,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/bf2ff10742b848baa3e39acc2c14d5d3, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 19:11:59,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/.tmp/cf2/f7bf0c8258c844c6955eaf3283b45e59 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/f7bf0c8258c844c6955eaf3283b45e59 2023-07-12 19:11:59,099 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/f7bf0c8258c844c6955eaf3283b45e59, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 19:11:59,100 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 68e5e51c22aa80fd52c4d2a0652112a3 in 48ms, sequenceid=82, compaction requested=true 2023-07-12 19:11:59,104 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-12 19:11:59,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:11:59,106 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. 2023-07-12 19:11:59,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 68e5e51c22aa80fd52c4d2a0652112a3: 2023-07-12 19:11:59,109 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,109 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=68e5e51c22aa80fd52c4d2a0652112a3, regionState=CLOSED 2023-07-12 19:11:59,110 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189119109"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189119109"}]},"ts":"1689189119109"} 2023-07-12 19:11:59,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-07-12 19:11:59,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 in 212 msec 2023-07-12 19:11:59,115 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-07-12 19:11:59,115 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN in 218 msec 2023-07-12 19:11:59,120 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:11:59,120 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:59,120 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:11:59,120 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:11:59,145 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:59,145 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:11:59,145 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:11:59,145 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:11:59,145 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:11:59,145 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:11:59,146 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:11:59,146 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:11:59,146 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:11:59,147 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 19:11:59,149 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 19:11:59,149 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:11:59,149 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:11:59,149 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:11:59,150 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:11:59,150 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:11:59,150 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:11:59,151 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:11:59,151 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:11:59,151 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:11:59,151 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:11:59,152 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:11:59,152 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:11:59,152 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:11:59,152 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:11:59,152 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:11:59,153 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:11:59,153 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:11:59,153 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:11:59,153 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:11:59,154 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:11:59,154 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb 2023-07-12 19:11:59,154 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:11:59,154 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:11:59,154 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:11:59,155 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:11:59,155 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:11:59,155 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:11:59,155 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:59,156 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:59,156 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,38353,1689189020667, table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 2023-07-12 19:11:59,156 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:59,156 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:59,156 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:59,157 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:59,157 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:59,157 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:11:59,157 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:59,158 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN 2023-07-12 19:11:59,158 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:59,158 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:11:59,158 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:59,158 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:59,158 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:59,158 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:11:59,158 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:11:59,158 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-07-12 19:11:59,159 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:11:59,160 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:11:59,160 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189149160 2023-07-12 19:11:59,161 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:11:59,162 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:59,164 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:11:59,164 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=68e5e51c22aa80fd52c4d2a0652112a3, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:11:59,164 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:11:59,164 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189179164 2023-07-12 19:11:59,164 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189239164 2023-07-12 19:11:59,164 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 19:11:59,167 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:11:59,178 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=68e5e51c22aa80fd52c4d2a0652112a3, threads=14 2023-07-12 19:11:59,178 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/108649e022dc4c9bb30deaf5bd890260 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,178 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/63323db36ea8495c8b004cfce8d8176d for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,178 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/99cba98576fe41958f05e66f4613c320 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,178 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/9a19ab58e24f463086c2367aea7becd5 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,179 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/bae3f6c2bf974087be57a28f2b9b6524 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,179 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/c9a72d8500a24cc7876cef4a7572be85 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,179 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/f7bf0c8258c844c6955eaf3283b45e59 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,179 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/2187000c9c5d477c998f64a30937a0a9 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,179 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/229a06affadc465bad0eddfa6f69d629 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,179 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/81615d3e65754dcd84f6b452c189d7ff for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,179 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/99c53f93030b456abb8c93f19343df11 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,179 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/bf2ff10742b848baa3e39acc2c14d5d3 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,180 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/d0bcf72680864ebc9e117386fec224ee for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,180 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/f8077d8e89bc4261bb003b6ad0bda3b1 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,312 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 19:11:59,319 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/c9a72d8500a24cc7876cef4a7572be85 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,356 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/99c53f93030b456abb8c93f19343df11 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,359 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/2187000c9c5d477c998f64a30937a0a9 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,384 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/bae3f6c2bf974087be57a28f2b9b6524 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,384 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/bf2ff10742b848baa3e39acc2c14d5d3 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,385 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/81615d3e65754dcd84f6b452c189d7ff for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,385 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/9a19ab58e24f463086c2367aea7becd5 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,385 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/63323db36ea8495c8b004cfce8d8176d for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,391 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/f7bf0c8258c844c6955eaf3283b45e59 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,392 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/99cba98576fe41958f05e66f4613c320 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,408 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/f8077d8e89bc4261bb003b6ad0bda3b1 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,408 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/229a06affadc465bad0eddfa6f69d629 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,683 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/108649e022dc4c9bb30deaf5bd890260 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,719 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/d0bcf72680864ebc9e117386fec224ee for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,719 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 68e5e51c22aa80fd52c4d2a0652112a3 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 19:11:59,764 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:11:59,765 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:59,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:11:59,765 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:11:59,914 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:59,915 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:11:59,915 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:11:59,915 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:11:59,916 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:11:59,916 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:11:59,916 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:11:59,916 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:11:59,916 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:11:59,918 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:11:59,922 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-12 19:11:59,923 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:11:59,923 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:11:59,924 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:11:59,924 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:11:59,924 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:11:59,924 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:11:59,925 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:11:59,925 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:11:59,925 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:11:59,925 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:11:59,926 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:11:59,926 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:11:59,926 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:11:59,926 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:11:59,927 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:11:59,927 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:11:59,927 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:11:59,927 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:11:59,927 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:11:59,927 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:11:59,927 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb 2023-07-12 19:11:59,928 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:11:59,928 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:11:59,928 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:11:59,928 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:11:59,928 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:11:59,928 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:11:59,929 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:59,929 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:59,929 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,38353,1689189020667, table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 2023-07-12 19:11:59,929 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:59,929 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:11:59,930 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:59,930 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:59,930 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:11:59,930 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:11:59,931 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:59,931 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN 2023-07-12 19:11:59,931 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:11:59,931 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:11:59,931 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:59,931 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:59,931 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:59,931 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:11:59,931 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:11:59,932 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 13 msec 2023-07-12 19:11:59,932 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:11:59,933 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:11:59,933 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189149933 2023-07-12 19:11:59,934 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:11:59,935 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:11:59,937 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:11:59,937 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=68e5e51c22aa80fd52c4d2a0652112a3, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:11:59,937 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:11:59,937 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189179937 2023-07-12 19:11:59,937 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189239937 2023-07-12 19:11:59,938 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:11:59,946 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=68e5e51c22aa80fd52c4d2a0652112a3, threads=14 2023-07-12 19:11:59,947 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/108649e022dc4c9bb30deaf5bd890260 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,947 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/63323db36ea8495c8b004cfce8d8176d for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,947 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/99cba98576fe41958f05e66f4613c320 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,947 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/9a19ab58e24f463086c2367aea7becd5 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,947 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/bae3f6c2bf974087be57a28f2b9b6524 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,947 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/c9a72d8500a24cc7876cef4a7572be85 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,947 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/f7bf0c8258c844c6955eaf3283b45e59 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,947 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/2187000c9c5d477c998f64a30937a0a9 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,947 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/229a06affadc465bad0eddfa6f69d629 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,947 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/81615d3e65754dcd84f6b452c189d7ff for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,947 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/99c53f93030b456abb8c93f19343df11 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,948 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/bf2ff10742b848baa3e39acc2c14d5d3 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,948 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/f8077d8e89bc4261bb003b6ad0bda3b1 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:11:59,948 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/d0bcf72680864ebc9e117386fec224ee for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:00,022 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/99c53f93030b456abb8c93f19343df11 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:00,023 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/d0bcf72680864ebc9e117386fec224ee for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:00,032 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/108649e022dc4c9bb30deaf5bd890260 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:00,032 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/c9a72d8500a24cc7876cef4a7572be85 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:00,032 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/bf2ff10742b848baa3e39acc2c14d5d3 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:00,036 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/99cba98576fe41958f05e66f4613c320 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:00,037 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/229a06affadc465bad0eddfa6f69d629 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:00,037 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/9a19ab58e24f463086c2367aea7becd5 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:00,037 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/81615d3e65754dcd84f6b452c189d7ff for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:00,037 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/bae3f6c2bf974087be57a28f2b9b6524 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:00,039 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/f7bf0c8258c844c6955eaf3283b45e59 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:00,039 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/2187000c9c5d477c998f64a30937a0a9 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:00,420 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/f8077d8e89bc4261bb003b6ad0bda3b1 for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:00,420 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/63323db36ea8495c8b004cfce8d8176d for region: 68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:00,420 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 68e5e51c22aa80fd52c4d2a0652112a3 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 19:12:00,460 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:12:00,468 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 19:12:00,470 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 19:12:00,471 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:12:00,471 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:00,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:12:00,471 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:12:00,624 ERROR [BP-1020377755-148.251.75.209-1689189016963 heartbeating to localhost.localdomain/127.0.0.1:43855] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1020377755-148.251.75.209-1689189016963 (Datanode Uuid 9225ce29-b06f-40b1-aaea-24784a80c6f4) service to localhost.localdomain/127.0.0.1:43855 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-12 19:12:00,688 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:00,688 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:12:00,688 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:12:00,689 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:12:00,689 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:12:00,689 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:12:00,690 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:12:00,690 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:12:00,690 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:12:00,692 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:12:00,695 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-12 19:12:00,695 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:12:00,695 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:12:00,695 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:12:00,695 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:12:00,695 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:12:00,695 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:12:00,696 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:00,696 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:12:00,696 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:00,696 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:00,696 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:00,696 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:12:00,696 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:00,697 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:00,697 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:00,697 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:12:00,697 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:12:00,697 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:00,697 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:00,698 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:00,698 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb 2023-07-12 19:12:00,698 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:00,698 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:00,698 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:00,698 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:12:00,698 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:00,698 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:00,699 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:00,699 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:00,699 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,38353,1689189020667, table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 2023-07-12 19:12:00,699 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:00,699 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:00,699 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:00,700 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:00,700 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:00,700 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:12:00,700 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:00,700 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN 2023-07-12 19:12:00,700 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:00,700 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:12:00,700 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:00,700 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:00,700 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:00,701 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:00,701 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:00,701 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-12 19:12:00,701 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:12:00,702 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:12:00,702 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189150702 2023-07-12 19:12:00,703 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:12:00,704 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:00,706 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:12:00,706 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=68e5e51c22aa80fd52c4d2a0652112a3, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:12:00,706 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:12:00,706 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189180706 2023-07-12 19:12:00,706 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189240706 2023-07-12 19:12:00,706 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:12:00,709 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:12:00,711 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:12:00,711 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:00,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:12:00,711 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:12:00,957 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:00,957 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:12:00,957 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:12:00,957 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:12:00,958 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:12:00,958 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:12:00,959 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:12:00,959 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:12:00,959 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:12:00,961 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:12:00,964 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-12 19:12:00,965 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:12:00,965 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:12:00,965 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:12:00,966 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:12:00,967 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:12:00,967 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:12:00,968 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:00,968 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:12:00,968 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:00,969 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:00,970 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:00,970 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:12:00,970 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:00,970 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:00,971 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:00,971 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:12:00,971 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:12:00,971 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:00,971 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:00,971 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:00,971 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb 2023-07-12 19:12:00,971 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:00,971 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:00,972 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:00,972 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:12:00,972 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:00,972 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:00,972 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:00,972 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:00,972 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,38353,1689189020667, table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 2023-07-12 19:12:00,972 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:00,973 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:00,973 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:00,973 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:00,973 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:00,973 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:12:00,973 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:00,973 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN 2023-07-12 19:12:00,974 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:00,974 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:12:00,974 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:00,974 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:00,974 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:00,974 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:00,974 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:00,974 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 13 msec 2023-07-12 19:12:00,975 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:12:00,976 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:12:00,976 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189150976 2023-07-12 19:12:00,977 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:12:00,977 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:00,980 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:12:00,980 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=68e5e51c22aa80fd52c4d2a0652112a3, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:12:00,980 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:12:00,981 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189180981 2023-07-12 19:12:00,981 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189240981 2023-07-12 19:12:00,981 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:12:00,981 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:12:00,983 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189120983"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1689189120983"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1689189120983"}]},"ts":"1689189120983"} 2023-07-12 19:12:00,983 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689189120983"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189120983"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189120983"}]},"ts":"1689189120983"} 2023-07-12 19:12:00,983 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689189120983"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189120983"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189120983"}]},"ts":"1689189120983"} 2023-07-12 19:12:00,985 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:12:00,985 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:00,985 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:12:00,985 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:12:01,231 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,231 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:12:01,232 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:12:01,232 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:12:01,232 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:12:01,233 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:12:01,233 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:12:01,233 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:12:01,233 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:12:01,235 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:12:01,238 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-07-12 19:12:01,239 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:12:01,239 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:12:01,239 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:12:01,239 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:12:01,239 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:12:01,239 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:12:01,240 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:01,240 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:12:01,240 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:01,240 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:01,240 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:01,240 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:12:01,241 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:01,241 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:01,241 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:01,241 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:12:01,241 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:12:01,242 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:01,242 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:01,242 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:01,242 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb 2023-07-12 19:12:01,242 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:01,242 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:01,243 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:01,243 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:12:01,243 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:01,243 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:01,243 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:01,244 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:01,244 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,38353,1689189020667, table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 2023-07-12 19:12:01,244 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:01,244 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:01,244 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:01,244 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:01,244 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:01,245 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:12:01,245 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,245 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN 2023-07-12 19:12:01,245 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:01,245 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:12:01,245 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,245 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,245 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,245 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:01,245 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:01,246 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-12 19:12:01,246 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:12:01,247 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:12:01,247 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189151247 2023-07-12 19:12:01,249 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,249 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:12:01,251 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:12:01,251 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=68e5e51c22aa80fd52c4d2a0652112a3, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:12:01,251 INFO [Listener at localhost.localdomain/40405] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,33255,1689189020635, table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:01,251 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=5a268511bb691df8adc4d725626a57c8, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-12 19:12:01,251 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=57d743025bac13dc49ba7f26600182c0, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-12 19:12:01,251 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:12:01,251 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189181251 2023-07-12 19:12:01,252 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189241252 2023-07-12 19:12:01,252 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 19:12:01,252 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689189121252"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1689189121252"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1689189121252"}]},"ts":"1689189121252"} 2023-07-12 19:12:01,252 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689189121252"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189121252"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189121252"}]},"ts":"1689189121252"} 2023-07-12 19:12:01,252 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689189121252"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189121252"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189121252"}]},"ts":"1689189121252"} 2023-07-12 19:12:01,254 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:12:01,256 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:12:01,256 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:12:01,256 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:12:01,502 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,502 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:12:01,502 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:12:01,503 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:12:01,503 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:12:01,504 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:12:01,504 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:12:01,504 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:12:01,504 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:12:01,506 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:12:01,508 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-07-12 19:12:01,508 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:12:01,508 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:12:01,509 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:12:01,509 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:12:01,509 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:12:01,509 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:12:01,510 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:01,510 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:12:01,511 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:01,511 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:01,511 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:01,511 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:12:01,512 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:01,512 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:01,512 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:01,513 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:12:01,513 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:12:01,513 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:01,513 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:01,514 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:01,514 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb 2023-07-12 19:12:01,514 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:01,514 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:01,515 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:01,515 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:12:01,515 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:01,515 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:01,515 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:01,515 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:01,516 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,38353,1689189020667, table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 2023-07-12 19:12:01,516 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:01,516 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:01,516 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:01,516 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:01,516 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:01,517 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:12:01,517 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,517 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN 2023-07-12 19:12:01,517 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:01,517 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:12:01,517 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,517 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,518 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,518 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:01,518 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:01,518 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-07-12 19:12:01,519 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:12:01,519 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:12:01,519 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189151519 2023-07-12 19:12:01,519 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:12:01,521 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,523 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:12:01,523 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=68e5e51c22aa80fd52c4d2a0652112a3, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:12:01,523 INFO [Listener at localhost.localdomain/40405] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,33255,1689189020635, table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:01,524 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=5a268511bb691df8adc4d725626a57c8, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-12 19:12:01,524 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=57d743025bac13dc49ba7f26600182c0, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-12 19:12:01,524 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:12:01,524 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189181524 2023-07-12 19:12:01,524 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189241524 2023-07-12 19:12:01,524 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 19:12:01,524 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:12:01,527 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5a268511bb691df8adc4d725626a57c8, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57d743025bac13dc49ba7f26600182c0, ASSIGN}] 2023-07-12 19:12:01,527 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:12:01,527 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,527 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:12:01,527 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:12:01,774 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,775 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:12:01,775 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:12:01,775 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:12:01,775 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:12:01,776 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:12:01,776 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:12:01,776 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:12:01,777 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:12:01,778 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 19:12:01,782 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-07-12 19:12:01,782 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:12:01,782 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:12:01,783 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:12:01,784 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:12:01,784 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:12:01,785 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:12:01,785 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:01,785 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:12:01,785 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:01,786 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:01,786 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:01,786 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:12:01,786 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:01,786 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:01,786 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:01,786 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:12:01,786 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:12:01,787 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:01,787 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:01,787 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:01,787 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb 2023-07-12 19:12:01,787 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:01,787 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:01,787 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:01,787 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:12:01,787 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:01,787 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:01,788 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:01,788 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:01,788 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,38353,1689189020667, table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 2023-07-12 19:12:01,788 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:01,788 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:01,788 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:01,788 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:01,788 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:01,788 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:12:01,789 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,789 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN 2023-07-12 19:12:01,789 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:01,789 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:12:01,789 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,789 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,789 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,789 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:01,789 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:01,789 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-12 19:12:01,790 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:12:01,790 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:12:01,791 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189151791 2023-07-12 19:12:01,792 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:12:01,792 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:01,794 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:12:01,794 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=68e5e51c22aa80fd52c4d2a0652112a3, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:12:01,794 INFO [Listener at localhost.localdomain/40405] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,33255,1689189020635, table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:01,795 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=5a268511bb691df8adc4d725626a57c8, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-12 19:12:01,795 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=57d743025bac13dc49ba7f26600182c0, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-12 19:12:01,795 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:12:01,795 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189181795 2023-07-12 19:12:01,795 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189241795 2023-07-12 19:12:01,795 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 19:12:01,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5a268511bb691df8adc4d725626a57c8, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57d743025bac13dc49ba7f26600182c0, ASSIGN}] 2023-07-12 19:12:01,795 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:12:01,797 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57d743025bac13dc49ba7f26600182c0, ASSIGN 2023-07-12 19:12:01,798 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57d743025bac13dc49ba7f26600182c0, ASSIGN; state=CLOSED, location=jenkins-hbase20.apache.org,33255,1689189020635; forceNewPlan=false, retain=false 2023-07-12 19:12:01,799 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5a268511bb691df8adc4d725626a57c8, ASSIGN 2023-07-12 19:12:01,799 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5a268511bb691df8adc4d725626a57c8, ASSIGN; state=CLOSED, location=jenkins-hbase20.apache.org,33255,1689189020635; forceNewPlan=false, retain=false 2023-07-12 19:12:01,948 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-12 19:12:01,950 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=5a268511bb691df8adc4d725626a57c8, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:01,950 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689189121950"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189121950"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189121950"}]},"ts":"1689189121950"} 2023-07-12 19:12:01,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure 5a268511bb691df8adc4d725626a57c8, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:12:01,954 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=57d743025bac13dc49ba7f26600182c0, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:01,954 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689189121954"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189121954"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189121954"}]},"ts":"1689189121954"} 2023-07-12 19:12:01,955 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 57d743025bac13dc49ba7f26600182c0, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:12:02,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8. 2023-07-12 19:12:02,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5a268511bb691df8adc4d725626a57c8, NAME => 'testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.', STARTKEY => '', ENDKEY => '41'} 2023-07-12 19:12:02,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:12:02,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 5a268511bb691df8adc4d725626a57c8 2023-07-12 19:12:02,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:12:02,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 5a268511bb691df8adc4d725626a57c8 2023-07-12 19:12:02,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 5a268511bb691df8adc4d725626a57c8 2023-07-12 19:12:02,114 INFO [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5a268511bb691df8adc4d725626a57c8 2023-07-12 19:12:02,115 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1 2023-07-12 19:12:02,115 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1 2023-07-12 19:12:02,116 INFO [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5a268511bb691df8adc4d725626a57c8 columnFamilyName cf1 2023-07-12 19:12:02,133 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/2187000c9c5d477c998f64a30937a0a9-bottom 2023-07-12 19:12:02,139 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/229a06affadc465bad0eddfa6f69d629-bottom 2023-07-12 19:12:02,145 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/81615d3e65754dcd84f6b452c189d7ff-bottom 2023-07-12 19:12:02,150 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/99c53f93030b456abb8c93f19343df11-bottom 2023-07-12 19:12:02,155 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/bf2ff10742b848baa3e39acc2c14d5d3-bottom 2023-07-12 19:12:02,160 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/d0bcf72680864ebc9e117386fec224ee-bottom 2023-07-12 19:12:02,165 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/f8077d8e89bc4261bb003b6ad0bda3b1-bottom 2023-07-12 19:12:02,165 INFO [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(310): Store=5a268511bb691df8adc4d725626a57c8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:02,165 INFO [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5a268511bb691df8adc4d725626a57c8 2023-07-12 19:12:02,166 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2 2023-07-12 19:12:02,166 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2 2023-07-12 19:12:02,167 INFO [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5a268511bb691df8adc4d725626a57c8 columnFamilyName cf2 2023-07-12 19:12:02,181 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/108649e022dc4c9bb30deaf5bd890260-bottom 2023-07-12 19:12:02,185 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/63323db36ea8495c8b004cfce8d8176d-bottom 2023-07-12 19:12:02,190 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/99cba98576fe41958f05e66f4613c320-bottom 2023-07-12 19:12:02,195 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/9a19ab58e24f463086c2367aea7becd5-bottom 2023-07-12 19:12:02,200 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/bae3f6c2bf974087be57a28f2b9b6524-bottom 2023-07-12 19:12:02,205 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/c9a72d8500a24cc7876cef4a7572be85-bottom 2023-07-12 19:12:02,211 DEBUG [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/f7bf0c8258c844c6955eaf3283b45e59-bottom 2023-07-12 19:12:02,211 INFO [StoreOpener-5a268511bb691df8adc4d725626a57c8-1] regionserver.HStore(310): Store=5a268511bb691df8adc4d725626a57c8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:02,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8 2023-07-12 19:12:02,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8 2023-07-12 19:12:02,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:12:02,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 5a268511bb691df8adc4d725626a57c8 2023-07-12 19:12:02,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 5a268511bb691df8adc4d725626a57c8; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65634057, jitterRate=-0.02197633683681488}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:12:02,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 5a268511bb691df8adc4d725626a57c8: 2023-07-12 19:12:02,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8., pid=117, masterSystemTime=1689189122103 2023-07-12 19:12:02,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:12:02,219 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:12:02,219 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:12:02,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:12:02,219 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8. 2023-07-12 19:12:02,219 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8. 2023-07-12 19:12:02,219 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HStore(1912): 5a268511bb691df8adc4d725626a57c8/cf1 is initiating minor compaction (all files) 2023-07-12 19:12:02,219 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HStore(1912): 5a268511bb691df8adc4d725626a57c8/cf2 is initiating minor compaction (all files) 2023-07-12 19:12:02,219 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5a268511bb691df8adc4d725626a57c8/cf1 in testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8. 2023-07-12 19:12:02,219 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 5a268511bb691df8adc4d725626a57c8/cf2 in testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8. 2023-07-12 19:12:02,220 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/c9a72d8500a24cc7876cef4a7572be85-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/99cba98576fe41958f05e66f4613c320-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/bae3f6c2bf974087be57a28f2b9b6524-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/63323db36ea8495c8b004cfce8d8176d-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/108649e022dc4c9bb30deaf5bd890260-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/9a19ab58e24f463086c2367aea7becd5-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/f7bf0c8258c844c6955eaf3283b45e59-bottom] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/.tmp, totalSize=34.9 K 2023-07-12 19:12:02,220 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/f8077d8e89bc4261bb003b6ad0bda3b1-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/81615d3e65754dcd84f6b452c189d7ff-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/d0bcf72680864ebc9e117386fec224ee-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/99c53f93030b456abb8c93f19343df11-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/229a06affadc465bad0eddfa6f69d629-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/2187000c9c5d477c998f64a30937a0a9-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/bf2ff10742b848baa3e39acc2c14d5d3-bottom] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/.tmp, totalSize=34.9 K 2023-07-12 19:12:02,220 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689189116663 2023-07-12 19:12:02,220 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689189116663 2023-07-12 19:12:02,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8. 2023-07-12 19:12:02,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8. 2023-07-12 19:12:02,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0. 2023-07-12 19:12:02,221 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting 99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689189116842 2023-07-12 19:12:02,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 57d743025bac13dc49ba7f26600182c0, NAME => 'testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.', STARTKEY => '41', ENDKEY => ''} 2023-07-12 19:12:02,221 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting 81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689189116842 2023-07-12 19:12:02,221 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=5a268511bb691df8adc4d725626a57c8, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:02,221 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689189117027 2023-07-12 19:12:02,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:12:02,221 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689189122221"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189122221"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189122221"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189122221"}]},"ts":"1689189122221"} 2023-07-12 19:12:02,221 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689189117027 2023-07-12 19:12:02,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:02,221 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting 63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689189117225 2023-07-12 19:12:02,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:12:02,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:02,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:02,222 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting 99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689189117225 2023-07-12 19:12:02,222 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting 108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689189117405 2023-07-12 19:12:02,222 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting 9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689189117615 2023-07-12 19:12:02,222 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting 229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689189117405 2023-07-12 19:12:02,222 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689189117805 2023-07-12 19:12:02,223 INFO [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:02,223 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting 2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689189117615 2023-07-12 19:12:02,223 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689189117805 2023-07-12 19:12:02,223 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1 2023-07-12 19:12:02,223 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1 2023-07-12 19:12:02,224 INFO [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 57d743025bac13dc49ba7f26600182c0 columnFamilyName cf1 2023-07-12 19:12:02,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-07-12 19:12:02,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 5a268511bb691df8adc4d725626a57c8, server=jenkins-hbase20.apache.org,33255,1689189020635 in 270 msec 2023-07-12 19:12:02,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5a268511bb691df8adc4d725626a57c8, ASSIGN in 429 msec 2023-07-12 19:12:02,234 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] throttle.PressureAwareThroughputController(145): 5a268511bb691df8adc4d725626a57c8#cf2#compaction#106 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:12:02,238 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/2187000c9c5d477c998f64a30937a0a9-top 2023-07-12 19:12:02,238 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5a268511bb691df8adc4d725626a57c8#cf1#compaction#107 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:12:02,241 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/229a06affadc465bad0eddfa6f69d629-top 2023-07-12 19:12:02,245 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/81615d3e65754dcd84f6b452c189d7ff-top 2023-07-12 19:12:02,248 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/99c53f93030b456abb8c93f19343df11-top 2023-07-12 19:12:02,249 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/.tmp/cf2/a5110686d5804709b1f225e0651ceab0 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/a5110686d5804709b1f225e0651ceab0 2023-07-12 19:12:02,250 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/.tmp/cf1/2bd93b16028a4294854de47e6c863afd as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/2bd93b16028a4294854de47e6c863afd 2023-07-12 19:12:02,253 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5a268511bb691df8adc4d725626a57c8/cf2 of 5a268511bb691df8adc4d725626a57c8 into a5110686d5804709b1f225e0651ceab0(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:12:02,253 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5a268511bb691df8adc4d725626a57c8/cf1 of 5a268511bb691df8adc4d725626a57c8 into 2bd93b16028a4294854de47e6c863afd(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:12:02,253 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5a268511bb691df8adc4d725626a57c8: 2023-07-12 19:12:02,253 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5a268511bb691df8adc4d725626a57c8: 2023-07-12 19:12:02,253 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8., storeName=5a268511bb691df8adc4d725626a57c8/cf2, priority=9, startTime=1689189122219; duration=0sec 2023-07-12 19:12:02,253 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8., storeName=5a268511bb691df8adc4d725626a57c8/cf1, priority=9, startTime=1689189122218; duration=0sec 2023-07-12 19:12:02,253 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:12:02,253 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:12:02,254 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/bf2ff10742b848baa3e39acc2c14d5d3-top 2023-07-12 19:12:02,257 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/d0bcf72680864ebc9e117386fec224ee-top 2023-07-12 19:12:02,260 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/f8077d8e89bc4261bb003b6ad0bda3b1-top 2023-07-12 19:12:02,260 INFO [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(310): Store=57d743025bac13dc49ba7f26600182c0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:02,260 INFO [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:02,261 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2 2023-07-12 19:12:02,261 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2 2023-07-12 19:12:02,261 INFO [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 57d743025bac13dc49ba7f26600182c0 columnFamilyName cf2 2023-07-12 19:12:02,272 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/108649e022dc4c9bb30deaf5bd890260-top 2023-07-12 19:12:02,275 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/63323db36ea8495c8b004cfce8d8176d-top 2023-07-12 19:12:02,278 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/99cba98576fe41958f05e66f4613c320-top 2023-07-12 19:12:02,281 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/9a19ab58e24f463086c2367aea7becd5-top 2023-07-12 19:12:02,284 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/bae3f6c2bf974087be57a28f2b9b6524-top 2023-07-12 19:12:02,287 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/c9a72d8500a24cc7876cef4a7572be85-top 2023-07-12 19:12:02,290 DEBUG [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/f7bf0c8258c844c6955eaf3283b45e59-top 2023-07-12 19:12:02,290 INFO [StoreOpener-57d743025bac13dc49ba7f26600182c0-1] regionserver.HStore(310): Store=57d743025bac13dc49ba7f26600182c0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:02,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:02,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:02,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:12:02,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:02,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 57d743025bac13dc49ba7f26600182c0; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60879520, jitterRate=-0.09282445907592773}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:12:02,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 57d743025bac13dc49ba7f26600182c0: 2023-07-12 19:12:02,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0., pid=118, masterSystemTime=1689189122103 2023-07-12 19:12:02,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:12:02,296 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:12:02,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:12:02,296 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:12:02,296 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0. 2023-07-12 19:12:02,296 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HStore(1912): 57d743025bac13dc49ba7f26600182c0/cf1 is initiating minor compaction (all files) 2023-07-12 19:12:02,296 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0. 2023-07-12 19:12:02,296 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HStore(1912): 57d743025bac13dc49ba7f26600182c0/cf2 is initiating minor compaction (all files) 2023-07-12 19:12:02,296 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 57d743025bac13dc49ba7f26600182c0/cf1 in testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0. 2023-07-12 19:12:02,296 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 57d743025bac13dc49ba7f26600182c0/cf2 in testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0. 2023-07-12 19:12:02,296 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/c9a72d8500a24cc7876cef4a7572be85-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/99cba98576fe41958f05e66f4613c320-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/bae3f6c2bf974087be57a28f2b9b6524-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/63323db36ea8495c8b004cfce8d8176d-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/108649e022dc4c9bb30deaf5bd890260-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/9a19ab58e24f463086c2367aea7becd5-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/f7bf0c8258c844c6955eaf3283b45e59-top] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/.tmp, totalSize=34.9 K 2023-07-12 19:12:02,296 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/f8077d8e89bc4261bb003b6ad0bda3b1-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/81615d3e65754dcd84f6b452c189d7ff-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/d0bcf72680864ebc9e117386fec224ee-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/99c53f93030b456abb8c93f19343df11-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/229a06affadc465bad0eddfa6f69d629-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/2187000c9c5d477c998f64a30937a0a9-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/bf2ff10742b848baa3e39acc2c14d5d3-top] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/.tmp, totalSize=34.9 K 2023-07-12 19:12:02,297 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689189116663 2023-07-12 19:12:02,297 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689189116663 2023-07-12 19:12:02,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0. 2023-07-12 19:12:02,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0. 2023-07-12 19:12:02,297 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting 99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689189116842 2023-07-12 19:12:02,297 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting 81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689189116842 2023-07-12 19:12:02,297 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=57d743025bac13dc49ba7f26600182c0, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:02,298 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689189117027 2023-07-12 19:12:02,298 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689189122297"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189122297"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189122297"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189122297"}]},"ts":"1689189122297"} 2023-07-12 19:12:02,298 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689189117027 2023-07-12 19:12:02,298 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting 63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689189117225 2023-07-12 19:12:02,298 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting 99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689189117225 2023-07-12 19:12:02,298 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting 108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1689189117405 2023-07-12 19:12:02,298 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting 229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1689189117405 2023-07-12 19:12:02,298 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting 9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1689189117615 2023-07-12 19:12:02,299 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting 2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1689189117615 2023-07-12 19:12:02,299 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689189117805 2023-07-12 19:12:02,299 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689189117805 2023-07-12 19:12:02,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-07-12 19:12:02,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 57d743025bac13dc49ba7f26600182c0, server=jenkins-hbase20.apache.org,33255,1689189020635 in 345 msec 2023-07-12 19:12:02,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=116, resume processing ppid=112 2023-07-12 19:12:02,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57d743025bac13dc49ba7f26600182c0, ASSIGN in 505 msec 2023-07-12 19:12:02,302 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 19:12:02,302 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:02,302 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:12:02,302 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 19:12:02,306 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] throttle.PressureAwareThroughputController(145): 57d743025bac13dc49ba7f26600182c0#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:12:02,306 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] throttle.PressureAwareThroughputController(145): 57d743025bac13dc49ba7f26600182c0#cf2#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:12:02,317 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/.tmp/cf2/abdb7ef52ca848f4846e12759681c118 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/abdb7ef52ca848f4846e12759681c118 2023-07-12 19:12:02,317 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/.tmp/cf1/ecdda72a6bf841b3926ec22be2930c2d as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/ecdda72a6bf841b3926ec22be2930c2d 2023-07-12 19:12:02,321 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 57d743025bac13dc49ba7f26600182c0/cf1 of 57d743025bac13dc49ba7f26600182c0 into ecdda72a6bf841b3926ec22be2930c2d(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:12:02,321 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 57d743025bac13dc49ba7f26600182c0/cf2 of 57d743025bac13dc49ba7f26600182c0 into abdb7ef52ca848f4846e12759681c118(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:12:02,321 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 57d743025bac13dc49ba7f26600182c0: 2023-07-12 19:12:02,321 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 57d743025bac13dc49ba7f26600182c0: 2023-07-12 19:12:02,321 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0., storeName=57d743025bac13dc49ba7f26600182c0/cf1, priority=9, startTime=1689189122295; duration=0sec 2023-07-12 19:12:02,321 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0., storeName=57d743025bac13dc49ba7f26600182c0/cf2, priority=9, startTime=1689189122296; duration=0sec 2023-07-12 19:12:02,321 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:12:02,321 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:12:02,546 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:02,546 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 19:12:02,546 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:12:02,546 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:12:02,546 ERROR [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 19:12:02,546 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 19:12:02,546 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 19:12:02,546 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 19:12:02,547 INFO [Listener at localhost.localdomain/40405] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 19:12:02,547 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 19:12:02,549 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-07-12 19:12:02,549 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 19:12:02,549 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 19:12:02,549 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 21] 2023-07-12 19:12:02,549 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-07-12 19:12:02,549 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 23] 2023-07-12 19:12:02,549 DEBUG [Listener at localhost.localdomain/40405] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-07-12 19:12:02,549 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 19:12:02,550 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 19:12:02,550 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 19:12:02,550 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 19:12:02,550 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:02,551 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=da5689e107dc59e64b9df0b3d4d1e069, daughterA=cae4263847f309e5972cfdde12d5c6ab, daughterB=4fb0a096d88532e6683c4be379845925 2023-07-12 19:12:02,551 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:02,551 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 19:12:02,551 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:02,551 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d0505bae67c35c958e3be134081a0fc8, daughterA=c0e9dac5a8ee81c6ad08618940a5f514, daughterB=18a1a0a446e655ef02aa7598aabe8703 2023-07-12 19:12:02,552 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:02,552 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 19:12:02,552 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:02,552 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=eb3ad527ad66eb4244896aace6f290b6, daughterA=7f35573e33b5dfbc24a5253b928fa5ba, daughterB=072cbd765c3d2a7e7a54842c70ef7f0e 2023-07-12 19:12:02,552 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=eb3ad527ad66eb4244896aace6f290b6, ASSIGN 2023-07-12 19:12:02,553 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:02,553 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 19:12:02,553 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:02,553 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=e841382c5b6af7d03ac7388d8251f64c, daughterA=54b1b528e1fad095bb10622ca6766109, daughterB=d6bc5978814690a86824058ca1615eeb 2023-07-12 19:12:02,553 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:02,553 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 19:12:02,553 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:02,553 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=0e61b0eec6c8d265f5fb01657ce1d2a0, daughterA=6328ad5feb88f12be2101cce567df742, daughterB=22bf8485475c754fcd294e2642c4de0e 2023-07-12 19:12:02,554 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:02,554 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 19:12:02,554 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:02,554 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:02,554 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,38353,1689189020667, table=testRollbackForSplitTableRegionWithReplica, region=a73a7d48997f0931e435d6a022fdfb50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=cdf8695334a5bdad2bdb8206cb54526f, daughterA=fb59e81299047e614692426751d6bb15, daughterB=f274217ecff0fbc2c89d6f6d92d92b21 2023-07-12 19:12:02,554 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:02,554 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 19:12:02,555 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:02,555 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:02,555 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 19:12:02,555 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:12:02,556 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:02,556 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN 2023-07-12 19:12:02,556 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:02,556 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5a268511bb691df8adc4d725626a57c8, ASSIGN 2023-07-12 19:12:02,556 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57d743025bac13dc49ba7f26600182c0, ASSIGN 2023-07-12 19:12:02,556 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 5a268511bb691df8adc4d725626a57c8, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:02,557 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 57d743025bac13dc49ba7f26600182c0, server=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:02,557 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 held the lock before restarting, call acquireLock to restore it. 2023-07-12 19:12:02,557 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:02,557 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:02,557 INFO [Listener at localhost.localdomain/40405] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:02,557 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:02,557 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 68e5e51c22aa80fd52c4d2a0652112a3, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:02,557 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5a268511bb691df8adc4d725626a57c8, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:02,557 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57d743025bac13dc49ba7f26600182c0, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:02,557 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 5a268511bb691df8adc4d725626a57c8, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:02,557 DEBUG [Listener at localhost.localdomain/40405] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 57d743025bac13dc49ba7f26600182c0, server=jenkins-hbase20.apache.org,33255,1689189020635 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 19:12:02,557 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-07-12 19:12:02,558 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 19:12:02,558 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667, table=hbase:meta, region=1588230740 2023-07-12 19:12:02,559 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689189152559 2023-07-12 19:12:02,560 DEBUG [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 19:12:02,561 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:02,564 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=da4636671d8cf580eb751750528e08a9, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,45655,1689189020596, regionLocation=jenkins-hbase20.apache.org,45655,1689189020596, openSeqNum=2 2023-07-12 19:12:02,564 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=68e5e51c22aa80fd52c4d2a0652112a3, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=2 2023-07-12 19:12:02,564 INFO [Listener at localhost.localdomain/40405] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,33255,1689189020635, table=testRecoveryAndDoubleExecution, region=68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:02,564 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=5a268511bb691df8adc4d725626a57c8, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=86 2023-07-12 19:12:02,564 DEBUG [Listener at localhost.localdomain/40405] assignment.RegionStateStore(147): Load hbase:meta entry region=57d743025bac13dc49ba7f26600182c0, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,33255,1689189020635, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635, openSeqNum=86 2023-07-12 19:12:02,565 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 19:12:02,565 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689189182565 2023-07-12 19:12:02,565 INFO [Listener at localhost.localdomain/40405] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689189242565 2023-07-12 19:12:02,565 INFO [Listener at localhost.localdomain/40405] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 19:12:02,565 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 19:12:02,568 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=68e5e51c22aa80fd52c4d2a0652112a3, daughterA=5a268511bb691df8adc4d725626a57c8, daughterB=57d743025bac13dc49ba7f26600182c0 in 4.4910 sec 2023-07-12 19:12:02,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33255] ipc.CallRunner(144): callId: 858 service: ClientService methodName: Scan size: 154 connection: 148.251.75.209:56298 deadline: 1689189182817, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3. is not online on jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:02,941 DEBUG [Listener at localhost.localdomain/40405] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-12 19:12:02,950 INFO [Listener at localhost.localdomain/40405] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-12 19:12:02,951 DEBUG [Listener at localhost.localdomain/40405] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-12 19:12:02,958 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:12:02,960 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-07-12 19:12:02,960 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testRecoveryAndDoubleExecution 2023-07-12 19:12:02,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:12:02,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-07-12 19:12:02,963 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189122963"}]},"ts":"1689189122963"} 2023-07-12 19:12:02,964 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-07-12 19:12:02,965 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-07-12 19:12:02,966 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5a268511bb691df8adc4d725626a57c8, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57d743025bac13dc49ba7f26600182c0, UNASSIGN}] 2023-07-12 19:12:02,966 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57d743025bac13dc49ba7f26600182c0, UNASSIGN 2023-07-12 19:12:02,967 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=57d743025bac13dc49ba7f26600182c0, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:02,967 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689189122967"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189122967"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189122967"}]},"ts":"1689189122967"} 2023-07-12 19:12:02,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 57d743025bac13dc49ba7f26600182c0, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:12:02,970 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5a268511bb691df8adc4d725626a57c8, UNASSIGN 2023-07-12 19:12:02,970 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=5a268511bb691df8adc4d725626a57c8, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:02,970 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689189122970"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189122970"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189122970"}]},"ts":"1689189122970"} 2023-07-12 19:12:02,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 5a268511bb691df8adc4d725626a57c8, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:12:03,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-07-12 19:12:03,120 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 5a268511bb691df8adc4d725626a57c8 2023-07-12 19:12:03,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 5a268511bb691df8adc4d725626a57c8, disabling compactions & flushes 2023-07-12 19:12:03,120 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8. 2023-07-12 19:12:03,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8. 2023-07-12 19:12:03,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8. after waiting 0 ms 2023-07-12 19:12:03,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8. 2023-07-12 19:12:03,122 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/f8077d8e89bc4261bb003b6ad0bda3b1-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/81615d3e65754dcd84f6b452c189d7ff-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/d0bcf72680864ebc9e117386fec224ee-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/99c53f93030b456abb8c93f19343df11-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/229a06affadc465bad0eddfa6f69d629-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/2187000c9c5d477c998f64a30937a0a9-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/bf2ff10742b848baa3e39acc2c14d5d3-bottom] to archive 2023-07-12 19:12:03,124 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:12:03,128 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,130 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,132 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,133 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,135 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,136 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,138 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,139 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/c9a72d8500a24cc7876cef4a7572be85-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/99cba98576fe41958f05e66f4613c320-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/bae3f6c2bf974087be57a28f2b9b6524-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/63323db36ea8495c8b004cfce8d8176d-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/108649e022dc4c9bb30deaf5bd890260-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/9a19ab58e24f463086c2367aea7becd5-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/f7bf0c8258c844c6955eaf3283b45e59-bottom] to archive 2023-07-12 19:12:03,140 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:12:03,141 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,143 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,144 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,146 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,148 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,150 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,151 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,156 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 19:12:03,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:12:03,157 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8. 2023-07-12 19:12:03,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 5a268511bb691df8adc4d725626a57c8: 2023-07-12 19:12:03,159 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 5a268511bb691df8adc4d725626a57c8 2023-07-12 19:12:03,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:03,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 57d743025bac13dc49ba7f26600182c0, disabling compactions & flushes 2023-07-12 19:12:03,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0. 2023-07-12 19:12:03,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0. 2023-07-12 19:12:03,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0. after waiting 0 ms 2023-07-12 19:12:03,160 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=5a268511bb691df8adc4d725626a57c8, regionState=CLOSED 2023-07-12 19:12:03,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0. 2023-07-12 19:12:03,160 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689189123160"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189123160"}]},"ts":"1689189123160"} 2023-07-12 19:12:03,162 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/f8077d8e89bc4261bb003b6ad0bda3b1-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/81615d3e65754dcd84f6b452c189d7ff-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/d0bcf72680864ebc9e117386fec224ee-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/99c53f93030b456abb8c93f19343df11-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/229a06affadc465bad0eddfa6f69d629-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/2187000c9c5d477c998f64a30937a0a9-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/bf2ff10742b848baa3e39acc2c14d5d3-top] to archive 2023-07-12 19:12:03,163 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:12:03,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-07-12 19:12:03,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 5a268511bb691df8adc4d725626a57c8, server=jenkins-hbase20.apache.org,33255,1689189020635 in 191 msec 2023-07-12 19:12:03,165 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5a268511bb691df8adc4d725626a57c8, UNASSIGN in 199 msec 2023-07-12 19:12:03,167 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,168 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,168 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,169 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,170 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,171 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,173 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/c9a72d8500a24cc7876cef4a7572be85-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/99cba98576fe41958f05e66f4613c320-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/bae3f6c2bf974087be57a28f2b9b6524-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/63323db36ea8495c8b004cfce8d8176d-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/108649e022dc4c9bb30deaf5bd890260-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/9a19ab58e24f463086c2367aea7becd5-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/f7bf0c8258c844c6955eaf3283b45e59-top] to archive 2023-07-12 19:12:03,173 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:12:03,174 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,175 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,176 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,177 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,178 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,179 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,179 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,182 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 19:12:03,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:12:03,183 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0. 2023-07-12 19:12:03,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 57d743025bac13dc49ba7f26600182c0: 2023-07-12 19:12:03,185 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:03,186 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=57d743025bac13dc49ba7f26600182c0, regionState=CLOSED 2023-07-12 19:12:03,186 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689189123185"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189123185"}]},"ts":"1689189123185"} 2023-07-12 19:12:03,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-07-12 19:12:03,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 57d743025bac13dc49ba7f26600182c0, server=jenkins-hbase20.apache.org,33255,1689189020635 in 219 msec 2023-07-12 19:12:03,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-07-12 19:12:03,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57d743025bac13dc49ba7f26600182c0, UNASSIGN in 222 msec 2023-07-12 19:12:03,190 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189123190"}]},"ts":"1689189123190"} 2023-07-12 19:12:03,190 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-07-12 19:12:03,191 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-07-12 19:12:03,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 232 msec 2023-07-12 19:12:03,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-07-12 19:12:03,267 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-07-12 19:12:03,269 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testRecoveryAndDoubleExecution 2023-07-12 19:12:03,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:12:03,274 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:12:03,275 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:12:03,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-07-12 19:12:03,283 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,284 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8 2023-07-12 19:12:03,284 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:03,287 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/recovered.edits] 2023-07-12 19:12:03,287 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/recovered.edits] 2023-07-12 19:12:03,287 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/recovered.edits] 2023-07-12 19:12:03,292 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/.regioninfo to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/.regioninfo 2023-07-12 19:12:03,292 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/.regioninfo to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/.regioninfo 2023-07-12 19:12:03,293 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/2187000c9c5d477c998f64a30937a0a9 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/2187000c9c5d477c998f64a30937a0a9 2023-07-12 19:12:03,294 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/229a06affadc465bad0eddfa6f69d629 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/229a06affadc465bad0eddfa6f69d629 2023-07-12 19:12:03,295 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf1/2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf1/2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,296 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf1/2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf1/2187000c9c5d477c998f64a30937a0a9.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,296 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/81615d3e65754dcd84f6b452c189d7ff to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/81615d3e65754dcd84f6b452c189d7ff 2023-07-12 19:12:03,297 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf1/229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf1/229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,297 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf1/229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf1/229a06affadc465bad0eddfa6f69d629.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,297 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/99c53f93030b456abb8c93f19343df11 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/99c53f93030b456abb8c93f19343df11 2023-07-12 19:12:03,298 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf1/81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf1/81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,298 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf1/81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf1/81615d3e65754dcd84f6b452c189d7ff.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,299 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/bf2ff10742b848baa3e39acc2c14d5d3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/bf2ff10742b848baa3e39acc2c14d5d3 2023-07-12 19:12:03,300 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf1/99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf1/99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,300 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf1/99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf1/99c53f93030b456abb8c93f19343df11.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,301 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/d0bcf72680864ebc9e117386fec224ee to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/d0bcf72680864ebc9e117386fec224ee 2023-07-12 19:12:03,301 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf1/bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf1/bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,302 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf1/bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf1/bf2ff10742b848baa3e39acc2c14d5d3.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,302 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/f8077d8e89bc4261bb003b6ad0bda3b1 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf1/f8077d8e89bc4261bb003b6ad0bda3b1 2023-07-12 19:12:03,303 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf1/d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf1/d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,303 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf1/d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf1/d0bcf72680864ebc9e117386fec224ee.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,304 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf1/f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf1/f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,305 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf1/f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf1/f8077d8e89bc4261bb003b6ad0bda3b1.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,305 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/108649e022dc4c9bb30deaf5bd890260 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/108649e022dc4c9bb30deaf5bd890260 2023-07-12 19:12:03,307 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/63323db36ea8495c8b004cfce8d8176d to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/63323db36ea8495c8b004cfce8d8176d 2023-07-12 19:12:03,307 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf2/108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf2/108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,308 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf2/108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf2/108649e022dc4c9bb30deaf5bd890260.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,308 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/99cba98576fe41958f05e66f4613c320 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/99cba98576fe41958f05e66f4613c320 2023-07-12 19:12:03,309 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf2/63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf2/63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,309 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf2/63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf2/63323db36ea8495c8b004cfce8d8176d.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,309 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/9a19ab58e24f463086c2367aea7becd5 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/9a19ab58e24f463086c2367aea7becd5 2023-07-12 19:12:03,309 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf2/99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf2/99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,310 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf2/99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf2/99cba98576fe41958f05e66f4613c320.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,310 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/bae3f6c2bf974087be57a28f2b9b6524 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/bae3f6c2bf974087be57a28f2b9b6524 2023-07-12 19:12:03,311 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf2/9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf2/9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,311 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf2/9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf2/9a19ab58e24f463086c2367aea7becd5.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,311 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/c9a72d8500a24cc7876cef4a7572be85 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/c9a72d8500a24cc7876cef4a7572be85 2023-07-12 19:12:03,311 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf2/bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf2/bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,312 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf2/bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf2/bae3f6c2bf974087be57a28f2b9b6524.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,312 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/f7bf0c8258c844c6955eaf3283b45e59 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/cf2/f7bf0c8258c844c6955eaf3283b45e59 2023-07-12 19:12:03,313 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf2/c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf2/c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,313 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf2/c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf2/c9a72d8500a24cc7876cef4a7572be85.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,313 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf2/f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/5a268511bb691df8adc4d725626a57c8/cf2/f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,314 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf2/f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/57d743025bac13dc49ba7f26600182c0/cf2/f7bf0c8258c844c6955eaf3283b45e59.68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,314 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/recovered.edits/85.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3/recovered.edits/85.seqid 2023-07-12 19:12:03,315 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/68e5e51c22aa80fd52c4d2a0652112a3 2023-07-12 19:12:03,315 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/2bd93b16028a4294854de47e6c863afd to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf1/2bd93b16028a4294854de47e6c863afd 2023-07-12 19:12:03,316 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/ecdda72a6bf841b3926ec22be2930c2d to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf1/ecdda72a6bf841b3926ec22be2930c2d 2023-07-12 19:12:03,317 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/a5110686d5804709b1f225e0651ceab0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/cf2/a5110686d5804709b1f225e0651ceab0 2023-07-12 19:12:03,317 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/abdb7ef52ca848f4846e12759681c118 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/cf2/abdb7ef52ca848f4846e12759681c118 2023-07-12 19:12:03,319 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/recovered.edits/90.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8/recovered.edits/90.seqid 2023-07-12 19:12:03,320 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/recovered.edits/90.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0/recovered.edits/90.seqid 2023-07-12 19:12:03,320 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/5a268511bb691df8adc4d725626a57c8 2023-07-12 19:12:03,320 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testRecoveryAndDoubleExecution/57d743025bac13dc49ba7f26600182c0 2023-07-12 19:12:03,320 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-07-12 19:12:03,322 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:12:03,324 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-07-12 19:12:03,325 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-07-12 19:12:03,326 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:12:03,326 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-07-12 19:12:03,326 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189123326"}]},"ts":"9223372036854775807"} 2023-07-12 19:12:03,326 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189123326"}]},"ts":"9223372036854775807"} 2023-07-12 19:12:03,326 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189123326"}]},"ts":"9223372036854775807"} 2023-07-12 19:12:03,328 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-12 19:12:03,328 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 68e5e51c22aa80fd52c4d2a0652112a3, NAME => 'testRecoveryAndDoubleExecution,,1689189116158.68e5e51c22aa80fd52c4d2a0652112a3.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 5a268511bb691df8adc4d725626a57c8, NAME => 'testRecoveryAndDoubleExecution,,1689189118073.5a268511bb691df8adc4d725626a57c8.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 57d743025bac13dc49ba7f26600182c0, NAME => 'testRecoveryAndDoubleExecution,41,1689189118073.57d743025bac13dc49ba7f26600182c0.', STARTKEY => '41', ENDKEY => ''}] 2023-07-12 19:12:03,328 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-07-12 19:12:03,328 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689189123328"}]},"ts":"9223372036854775807"} 2023-07-12 19:12:03,329 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-07-12 19:12:03,330 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 19:12:03,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 61 msec 2023-07-12 19:12:03,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-07-12 19:12:03,378 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-07-12 19:12:03,399 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=376 (was 369) - Thread LEAK? -, OpenFileDescriptor=835 (was 745) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=93 (was 92) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=5736 (was 5782) 2023-07-12 19:12:03,414 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=376, OpenFileDescriptor=835, MaxFileDescriptor=60000, SystemLoadAverage=93, ProcessCount=170, AvailableMemoryMB=5735 2023-07-12 19:12:03,414 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:12:03,417 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 19:12:03,417 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 19:12:03,418 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,419 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 19:12:03,420 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 19:12:03,421 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:03,422 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564 empty. 2023-07-12 19:12:03,422 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:03,422 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-07-12 19:12:03,432 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-07-12 19:12:03,434 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 891651b57d0223c960e1a67ca99da564, NAME => 'testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp 2023-07-12 19:12:03,443 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:12:03,444 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1604): Closing 891651b57d0223c960e1a67ca99da564, disabling compactions & flushes 2023-07-12 19:12:03,444 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:03,444 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:03,444 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. after waiting 0 ms 2023-07-12 19:12:03,444 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:03,444 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:03,444 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 891651b57d0223c960e1a67ca99da564: 2023-07-12 19:12:03,446 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 19:12:03,448 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189123448"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189123448"}]},"ts":"1689189123448"} 2023-07-12 19:12:03,449 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 19:12:03,450 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 19:12:03,450 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189123450"}]},"ts":"1689189123450"} 2023-07-12 19:12:03,451 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-07-12 19:12:03,454 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 19:12:03,456 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 19:12:03,456 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 19:12:03,456 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 19:12:03,456 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 19:12:03,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=891651b57d0223c960e1a67ca99da564, ASSIGN}] 2023-07-12 19:12:03,458 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=891651b57d0223c960e1a67ca99da564, ASSIGN 2023-07-12 19:12:03,458 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=891651b57d0223c960e1a67ca99da564, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,33255,1689189020635; forceNewPlan=false, retain=false 2023-07-12 19:12:03,609 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 19:12:03,611 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=891651b57d0223c960e1a67ca99da564, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:03,611 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189123611"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189123611"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189123611"}]},"ts":"1689189123611"} 2023-07-12 19:12:03,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 891651b57d0223c960e1a67ca99da564, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:12:03,627 ERROR [BP-1020377755-148.251.75.209-1689189016963 heartbeating to localhost.localdomain/127.0.0.1:43855] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1020377755-148.251.75.209-1689189016963 (Datanode Uuid 1de994d2-d43c-471e-8302-bc00c9b9674d) service to localhost.localdomain/127.0.0.1:43855 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-12 19:12:03,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:03,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 891651b57d0223c960e1a67ca99da564, NAME => 'testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564.', STARTKEY => '', ENDKEY => ''} 2023-07-12 19:12:03,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:12:03,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:03,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:12:03,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:03,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:03,779 INFO [StoreOpener-891651b57d0223c960e1a67ca99da564-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:03,782 DEBUG [StoreOpener-891651b57d0223c960e1a67ca99da564-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1 2023-07-12 19:12:03,782 DEBUG [StoreOpener-891651b57d0223c960e1a67ca99da564-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1 2023-07-12 19:12:03,782 INFO [StoreOpener-891651b57d0223c960e1a67ca99da564-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 891651b57d0223c960e1a67ca99da564 columnFamilyName cf1 2023-07-12 19:12:03,783 INFO [StoreOpener-891651b57d0223c960e1a67ca99da564-1] regionserver.HStore(310): Store=891651b57d0223c960e1a67ca99da564/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:03,783 INFO [StoreOpener-891651b57d0223c960e1a67ca99da564-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:03,785 DEBUG [StoreOpener-891651b57d0223c960e1a67ca99da564-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2 2023-07-12 19:12:03,785 DEBUG [StoreOpener-891651b57d0223c960e1a67ca99da564-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2 2023-07-12 19:12:03,786 INFO [StoreOpener-891651b57d0223c960e1a67ca99da564-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 891651b57d0223c960e1a67ca99da564 columnFamilyName cf2 2023-07-12 19:12:03,786 INFO [StoreOpener-891651b57d0223c960e1a67ca99da564-1] regionserver.HStore(310): Store=891651b57d0223c960e1a67ca99da564/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:03,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:03,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:03,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:12:03,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:03,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 19:12:03,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 891651b57d0223c960e1a67ca99da564; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59600739, jitterRate=-0.11187978088855743}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:12:03,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 891651b57d0223c960e1a67ca99da564: 2023-07-12 19:12:03,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564., pid=127, masterSystemTime=1689189123768 2023-07-12 19:12:03,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:03,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:03,804 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=891651b57d0223c960e1a67ca99da564, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:03,804 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189123804"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189123804"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189123804"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189123804"}]},"ts":"1689189123804"} 2023-07-12 19:12:03,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-07-12 19:12:03,808 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 891651b57d0223c960e1a67ca99da564, server=jenkins-hbase20.apache.org,33255,1689189020635 in 191 msec 2023-07-12 19:12:03,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-07-12 19:12:03,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=891651b57d0223c960e1a67ca99da564, ASSIGN in 352 msec 2023-07-12 19:12:03,811 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 19:12:03,811 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189123811"}]},"ts":"1689189123811"} 2023-07-12 19:12:03,812 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-07-12 19:12:03,814 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 19:12:03,814 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 396 msec 2023-07-12 19:12:03,915 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-07-12 19:12:03,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:12:03,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,936 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-12 19:12:03,936 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:03,936 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-12 19:12:03,936 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-12 19:12:03,937 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,937 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,938 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:03,938 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:03,938 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:03,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:03,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:03,938 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:03,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:03,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:03,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:03,938 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,939 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:12:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:03,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,940 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:03,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,940 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,940 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,940 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,940 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,940 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:03,940 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:03,940 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:03,940 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:03,940 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:03,940 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:03,940 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:03,940 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:03,940 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:03,940 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:03,940 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:03,941 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:03,941 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:03,941 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:12:03,941 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:12:03,941 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:03,941 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. started... 2023-07-12 19:12:03,942 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 891651b57d0223c960e1a67ca99da564 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 19:12:03,942 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,943 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,943 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,943 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:03,943 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,943 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:03,952 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf1/58c24e6f62884a5fa818a63cb789ee99 2023-07-12 19:12:03,968 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf2/641e38f9379c4a1a923c7a7ff4f4c5b6 2023-07-12 19:12:03,974 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf1/58c24e6f62884a5fa818a63cb789ee99 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/58c24e6f62884a5fa818a63cb789ee99 2023-07-12 19:12:03,980 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/58c24e6f62884a5fa818a63cb789ee99, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:12:03,981 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf2/641e38f9379c4a1a923c7a7ff4f4c5b6 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/641e38f9379c4a1a923c7a7ff4f4c5b6 2023-07-12 19:12:03,988 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/641e38f9379c4a1a923c7a7ff4f4c5b6, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:12:03,989 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 891651b57d0223c960e1a67ca99da564 in 47ms, sequenceid=6, compaction requested=false 2023-07-12 19:12:03,990 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 891651b57d0223c960e1a67ca99da564: 2023-07-12 19:12:03,990 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:03,990 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:12:03,990 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:12:03,990 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:03,990 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:03,990 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:03,992 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,992 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:03,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:03,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:03,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:03,993 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,993 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:03,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:03,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:03,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:03,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:03,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:03,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:03,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:03,998 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-12 19:12:03,998 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5901d792[Count = 0] remaining members to acquire global barrier 2023-07-12 19:12:03,998 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-12 19:12:03,998 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,000 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,000 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,000 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,001 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,000 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,001 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:12:04,000 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,001 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,001 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,001 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:12:04,001 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,001 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,001 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:12:04,001 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:12:04,003 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,003 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,003 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,003 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,003 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,003 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,003 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,003 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,003 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,003 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:12:04,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:12:04,010 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-12 19:12:04,010 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:12:04,010 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:12:04,010 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,010 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:12:04,017 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,017 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,017 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,017 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,017 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,017 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,018 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,019 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,017 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,020 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,023 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,024 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,024 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,029 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,029 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,029 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,030 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-12 19:12:04,030 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:12:04,029 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,029 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,029 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,030 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,030 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,029 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,030 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:12:04,030 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,030 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,031 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,031 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:12:04,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,031 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,032 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,032 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,032 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,032 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,032 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,132 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:12:04,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:12:04,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:12:04,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:12:04,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,150 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-12 19:12:04,151 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,151 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-12 19:12:04,151 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-12 19:12:04,151 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,151 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,152 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,152 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,152 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,152 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,152 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,153 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,153 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,153 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 19:12:04,153 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,153 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 19:12:04,153 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:04,154 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,153 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,154 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:04,153 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,154 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:04,154 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:04,154 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:12:04,154 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:12:04,154 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,154 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:04,154 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:04,154 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:04,154 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:04,154 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:04,154 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:04,154 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. started... 2023-07-12 19:12:04,154 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 891651b57d0223c960e1a67ca99da564 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:12:04,155 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,155 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,155 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,155 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:04,155 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,155 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:04,162 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf1/835ce9b34032471f85a7e6927f9b410b 2023-07-12 19:12:04,174 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf2/a455f6af45424df8b315e7ceaef50dee 2023-07-12 19:12:04,179 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf1/835ce9b34032471f85a7e6927f9b410b as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/835ce9b34032471f85a7e6927f9b410b 2023-07-12 19:12:04,184 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/835ce9b34032471f85a7e6927f9b410b, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:12:04,185 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf2/a455f6af45424df8b315e7ceaef50dee as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/a455f6af45424df8b315e7ceaef50dee 2023-07-12 19:12:04,191 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/a455f6af45424df8b315e7ceaef50dee, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:12:04,192 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 891651b57d0223c960e1a67ca99da564 in 38ms, sequenceid=19, compaction requested=false 2023-07-12 19:12:04,192 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 891651b57d0223c960e1a67ca99da564: 2023-07-12 19:12:04,192 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:04,192 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:12:04,192 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:12:04,192 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,192 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:04,192 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:04,194 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,194 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,194 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,194 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:04,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-12 19:12:04,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@43345c32[Count = 0] remaining members to acquire global barrier 2023-07-12 19:12:04,197 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-12 19:12:04,197 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,198 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,198 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,198 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,198 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,198 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,198 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:12:04,198 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,198 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,198 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,198 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:12:04,199 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,199 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,199 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:12:04,198 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:12:04,199 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,199 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,199 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,200 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,200 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,200 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,200 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,200 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,200 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,200 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:12:04,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:12:04,202 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-12 19:12:04,202 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:12:04,202 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:12:04,202 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,202 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:12:04,203 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,203 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,203 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,203 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,203 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,203 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,203 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,203 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,204 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,204 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,205 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,205 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,205 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,207 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,207 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,207 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,207 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,207 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,207 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,207 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,207 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-12 19:12:04,207 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:12:04,207 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,207 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,207 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,208 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:12:04,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,208 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:12:04,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,208 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,208 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,208 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,208 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,208 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,208 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,208 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,208 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,308 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:12:04,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:12:04,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:12:04,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:12:04,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,331 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-12 19:12:04,331 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,331 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-12 19:12:04,331 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-12 19:12:04,332 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,332 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,333 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,333 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,333 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,333 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,333 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,333 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:12:04,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,334 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,334 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,334 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 19:12:04,334 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,334 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 19:12:04,334 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,334 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,334 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:04,334 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 19:12:04,334 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,334 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:04,335 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:04,335 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,335 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:04,335 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:04,335 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,335 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,335 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:04,335 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:04,335 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:04,335 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:04,335 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:04,335 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:12:04,335 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:12:04,335 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:04,335 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. started... 2023-07-12 19:12:04,335 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 891651b57d0223c960e1a67ca99da564 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:12:04,336 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,336 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,337 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,337 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:04,337 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,337 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:04,348 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf1/44604be4a352490c83f5eba7426442e1 2023-07-12 19:12:04,361 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf2/62464eabac8747bc8a1535b5d2926158 2023-07-12 19:12:04,365 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf1/44604be4a352490c83f5eba7426442e1 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/44604be4a352490c83f5eba7426442e1 2023-07-12 19:12:04,368 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/44604be4a352490c83f5eba7426442e1, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:12:04,369 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf2/62464eabac8747bc8a1535b5d2926158 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/62464eabac8747bc8a1535b5d2926158 2023-07-12 19:12:04,372 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/62464eabac8747bc8a1535b5d2926158, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:12:04,373 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 891651b57d0223c960e1a67ca99da564 in 38ms, sequenceid=32, compaction requested=true 2023-07-12 19:12:04,373 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 891651b57d0223c960e1a67ca99da564: 2023-07-12 19:12:04,373 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:04,373 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:12:04,373 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:12:04,373 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,373 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:04,373 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:04,374 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,374 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,374 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,374 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:04,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,376 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-12 19:12:04,376 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@664eb6ea[Count = 0] remaining members to acquire global barrier 2023-07-12 19:12:04,376 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-12 19:12:04,376 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,377 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,377 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,377 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,377 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,377 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,377 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,377 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:12:04,377 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,377 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:12:04,377 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,377 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,377 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:12:04,377 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,377 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:12:04,378 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,378 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,378 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,378 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,378 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,378 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,378 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,378 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,378 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,378 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:12:04,380 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:12:04,380 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-12 19:12:04,380 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:12:04,380 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:12:04,380 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,380 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:12:04,381 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,381 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,381 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,381 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,381 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,381 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,381 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,381 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,381 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,382 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,383 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,383 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,383 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,385 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,385 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,385 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,385 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,385 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-12 19:12:04,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,385 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,385 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,385 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,386 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:12:04,386 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,386 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,386 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,386 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,386 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,386 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:12:04,386 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,386 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:12:04,386 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,386 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,386 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,486 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:12:04,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:12:04,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:12:04,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:12:04,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,506 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-12 19:12:04,506 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,506 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-12 19:12:04,506 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-12 19:12:04,507 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,507 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,508 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,508 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,508 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,508 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,508 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,508 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:12:04,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:04,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:04,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,509 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:04,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,510 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 19:12:04,510 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,510 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:04,510 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 19:12:04,510 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 19:12:04,510 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,510 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,510 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,510 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:04,510 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,510 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:04,511 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:04,510 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:04,511 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,511 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:04,511 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:12:04,511 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:12:04,511 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:04,511 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:04,511 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:04,511 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:04,511 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:04,511 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. started... 2023-07-12 19:12:04,511 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 891651b57d0223c960e1a67ca99da564 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:12:04,512 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,512 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,513 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,513 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:04,513 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,513 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:04,522 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf1/1afec31ee33a4661bce993525e12afe7 2023-07-12 19:12:04,537 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf2/157582c3bdf8489eb50d651833c2fc9d 2023-07-12 19:12:04,540 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf1/1afec31ee33a4661bce993525e12afe7 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/1afec31ee33a4661bce993525e12afe7 2023-07-12 19:12:04,543 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/1afec31ee33a4661bce993525e12afe7, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:12:04,543 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf2/157582c3bdf8489eb50d651833c2fc9d as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/157582c3bdf8489eb50d651833c2fc9d 2023-07-12 19:12:04,546 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/157582c3bdf8489eb50d651833c2fc9d, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:12:04,547 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 891651b57d0223c960e1a67ca99da564 in 36ms, sequenceid=45, compaction requested=true 2023-07-12 19:12:04,547 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 891651b57d0223c960e1a67ca99da564: 2023-07-12 19:12:04,547 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:04,547 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:12:04,547 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:12:04,547 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,547 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:04,547 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:04,549 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,549 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,549 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,549 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:04,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,551 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-12 19:12:04,551 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73c3b3b5[Count = 0] remaining members to acquire global barrier 2023-07-12 19:12:04,551 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-12 19:12:04,551 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,552 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,552 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,552 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,552 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,552 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,552 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,552 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:12:04,552 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:12:04,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,552 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,553 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,553 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,553 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,553 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:12:04,553 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:12:04,553 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,553 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,553 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,554 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,554 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,554 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,554 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,554 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,554 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,554 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:12:04,557 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:12:04,557 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-12 19:12:04,557 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:12:04,557 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:12:04,557 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,557 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:12:04,558 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,558 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,558 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,558 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,558 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,558 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,558 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,559 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,561 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,561 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,563 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,564 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,564 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,567 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,567 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,568 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,567 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,567 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,567 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,568 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,568 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,568 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,568 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-12 19:12:04,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:12:04,568 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,568 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,568 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,569 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:12:04,569 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,569 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,569 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:12:04,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,569 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,569 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,569 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,608 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 19:12:04,669 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:12:04,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:12:04,684 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:12:04,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:12:04,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,686 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-12 19:12:04,686 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,686 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-12 19:12:04,686 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-12 19:12:04,686 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,686 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,687 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,687 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,687 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,687 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,688 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,688 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:12:04,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:04,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:04,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,689 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:04,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,689 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,689 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,690 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 19:12:04,690 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,690 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 19:12:04,690 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 19:12:04,690 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:04,690 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,690 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,690 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:04,690 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,690 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,690 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:04,690 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:04,690 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:04,690 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:04,690 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:12:04,690 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:12:04,690 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,690 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:04,690 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:04,690 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:04,690 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:04,690 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:04,691 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. started... 2023-07-12 19:12:04,691 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 891651b57d0223c960e1a67ca99da564 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:12:04,692 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,692 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,692 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,692 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:04,692 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,692 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:04,702 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf1/dccc9585783b48d9bdfc59dbb6c8ff6f 2023-07-12 19:12:04,718 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf2/558a75f660654fa0ac39ae52e8308d6b 2023-07-12 19:12:04,725 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf1/dccc9585783b48d9bdfc59dbb6c8ff6f as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/dccc9585783b48d9bdfc59dbb6c8ff6f 2023-07-12 19:12:04,731 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/dccc9585783b48d9bdfc59dbb6c8ff6f, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:12:04,732 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf2/558a75f660654fa0ac39ae52e8308d6b as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/558a75f660654fa0ac39ae52e8308d6b 2023-07-12 19:12:04,737 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/558a75f660654fa0ac39ae52e8308d6b, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:12:04,737 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 891651b57d0223c960e1a67ca99da564 in 46ms, sequenceid=58, compaction requested=true 2023-07-12 19:12:04,737 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 891651b57d0223c960e1a67ca99da564: 2023-07-12 19:12:04,738 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:04,738 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:12:04,738 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:12:04,738 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,738 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:04,738 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:04,739 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,739 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,739 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,739 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:04,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-12 19:12:04,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24a95718[Count = 0] remaining members to acquire global barrier 2023-07-12 19:12:04,741 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-12 19:12:04,741 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,742 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,742 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,742 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,742 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,742 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,742 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:12:04,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,742 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,742 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:12:04,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,742 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,742 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,742 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,742 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:12:04,742 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,742 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:12:04,743 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,743 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,743 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,743 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,743 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,743 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,743 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,743 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,743 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,743 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:12:04,745 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:12:04,745 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-12 19:12:04,745 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:12:04,745 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:12:04,745 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,745 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:12:04,746 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,746 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,746 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,746 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,746 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,746 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,746 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,747 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,748 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,748 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,750 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,750 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,751 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,754 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,754 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,754 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,754 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,754 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,754 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,754 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,754 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,754 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-12 19:12:04,755 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:12:04,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,754 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,754 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,755 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,755 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:12:04,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,755 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,754 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,755 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,755 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:12:04,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,755 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,755 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,756 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,756 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,855 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:12:04,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:12:04,878 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:12:04,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 19:12:04,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,879 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-12 19:12:04,879 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,879 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-12 19:12:04,879 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-12 19:12:04,880 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,880 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,881 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,881 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,881 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,881 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,882 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,882 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:12:04,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,882 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,882 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,882 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:04,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:04,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,883 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,883 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:04,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,883 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,883 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 19:12:04,883 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 19:12:04,883 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,883 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 19:12:04,883 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,883 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,883 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:04,883 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:04,883 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,883 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 19:12:04,883 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,883 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:04,883 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:04,883 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:04,884 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:04,883 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:04,884 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 19:12:04,884 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:04,884 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:04,884 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:12:04,884 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:12:04,884 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:04,884 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. started... 2023-07-12 19:12:04,884 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 891651b57d0223c960e1a67ca99da564 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:12:04,885 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,886 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,886 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,886 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:04,886 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,886 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:04,896 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf1/508cf61a06584a56b86fec08da71cf88 2023-07-12 19:12:04,912 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf2/2abe8349bc614a7c8691eb38d8e54b09 2023-07-12 19:12:04,916 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf1/508cf61a06584a56b86fec08da71cf88 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/508cf61a06584a56b86fec08da71cf88 2023-07-12 19:12:04,921 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/508cf61a06584a56b86fec08da71cf88, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:12:04,922 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf2/2abe8349bc614a7c8691eb38d8e54b09 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/2abe8349bc614a7c8691eb38d8e54b09 2023-07-12 19:12:04,928 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/2abe8349bc614a7c8691eb38d8e54b09, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:12:04,929 INFO [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 891651b57d0223c960e1a67ca99da564 in 45ms, sequenceid=71, compaction requested=true 2023-07-12 19:12:04,929 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 891651b57d0223c960e1a67ca99da564: 2023-07-12 19:12:04,929 DEBUG [rs(jenkins-hbase20.apache.org,33255,1689189020635)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:04,929 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:12:04,929 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:12:04,929 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,929 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 19:12:04,929 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 19:12:04,931 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,931 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,931 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,931 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:04,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,935 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-12 19:12:04,935 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@451d4a73[Count = 0] remaining members to acquire global barrier 2023-07-12 19:12:04,935 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-12 19:12:04,935 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,936 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,936 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,936 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,936 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,936 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,936 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:12:04,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,937 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,936 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,937 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:12:04,937 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,937 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:12:04,937 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 19:12:04,937 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 19:12:04,937 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:12:04,938 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,938 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,938 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,938 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,938 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,938 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,939 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,939 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 19:12:04,939 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,939 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 19:12:04,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,33255,1689189020635': 2023-07-12 19:12:04,944 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-12 19:12:04,944 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-12 19:12:04,944 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:12:04,944 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:12:04,944 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,944 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:12:04,954 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,954 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,954 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,954 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,954 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,954 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:04,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,954 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,955 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:04,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,956 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,957 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:04,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,960 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:04,961 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,962 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,965 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,965 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,965 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,965 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,965 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-12 19:12:04,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:12:04,965 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:04,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,965 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,965 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:04,965 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,965 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,966 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,966 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 19:12:04,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:04,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:04,965 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:04,966 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:12:04,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,966 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:04,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:04,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:04,967 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,967 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:04,967 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:04,967 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:04,967 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 19:12:05,066 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:12:05,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:12:05,085 DEBUG [Listener at localhost.localdomain/40405] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:05,086 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=891651b57d0223c960e1a67ca99da564, daughterA=337efae4f3995300de0db431c0f12eeb, daughterB=7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:05,087 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=891651b57d0223c960e1a67ca99da564, daughterA=337efae4f3995300de0db431c0f12eeb, daughterB=7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:05,088 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=891651b57d0223c960e1a67ca99da564, daughterA=337efae4f3995300de0db431c0f12eeb, daughterB=7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:05,088 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=891651b57d0223c960e1a67ca99da564, daughterA=337efae4f3995300de0db431c0f12eeb, daughterB=7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:05,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=891651b57d0223c960e1a67ca99da564, UNASSIGN}] 2023-07-12 19:12:05,090 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=891651b57d0223c960e1a67ca99da564, UNASSIGN 2023-07-12 19:12:05,091 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=891651b57d0223c960e1a67ca99da564, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:05,091 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189125091"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189125091"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189125091"}]},"ts":"1689189125091"} 2023-07-12 19:12:05,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 891651b57d0223c960e1a67ca99da564, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:12:05,246 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,246 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 891651b57d0223c960e1a67ca99da564, disabling compactions & flushes 2023-07-12 19:12:05,246 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:05,246 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:05,246 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. after waiting 0 ms 2023-07-12 19:12:05,246 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:05,247 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 891651b57d0223c960e1a67ca99da564 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-12 19:12:05,662 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf1/898384d558794c1587adce395ab212a8 2023-07-12 19:12:05,683 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf2/5eae4dfef7d54b8887a94366bfadf052 2023-07-12 19:12:05,688 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf1/898384d558794c1587adce395ab212a8 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/898384d558794c1587adce395ab212a8 2023-07-12 19:12:05,693 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/898384d558794c1587adce395ab212a8, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 19:12:05,694 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.tmp/cf2/5eae4dfef7d54b8887a94366bfadf052 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/5eae4dfef7d54b8887a94366bfadf052 2023-07-12 19:12:05,699 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/5eae4dfef7d54b8887a94366bfadf052, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 19:12:05,700 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 891651b57d0223c960e1a67ca99da564 in 454ms, sequenceid=82, compaction requested=true 2023-07-12 19:12:05,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-12 19:12:05,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:12:05,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. 2023-07-12 19:12:05,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 891651b57d0223c960e1a67ca99da564: 2023-07-12 19:12:05,707 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,708 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=891651b57d0223c960e1a67ca99da564, regionState=CLOSED 2023-07-12 19:12:05,708 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189125708"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189125708"}]},"ts":"1689189125708"} 2023-07-12 19:12:05,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-07-12 19:12:05,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 891651b57d0223c960e1a67ca99da564, server=jenkins-hbase20.apache.org,33255,1689189020635 in 616 msec 2023-07-12 19:12:05,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-07-12 19:12:05,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=891651b57d0223c960e1a67ca99da564, UNASSIGN in 622 msec 2023-07-12 19:12:05,724 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=891651b57d0223c960e1a67ca99da564, threads=14 2023-07-12 19:12:05,725 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/157582c3bdf8489eb50d651833c2fc9d for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,725 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/2abe8349bc614a7c8691eb38d8e54b09 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,725 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/558a75f660654fa0ac39ae52e8308d6b for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,725 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/5eae4dfef7d54b8887a94366bfadf052 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,725 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/62464eabac8747bc8a1535b5d2926158 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,725 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/641e38f9379c4a1a923c7a7ff4f4c5b6 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,725 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/a455f6af45424df8b315e7ceaef50dee for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,725 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/1afec31ee33a4661bce993525e12afe7 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,725 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/44604be4a352490c83f5eba7426442e1 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,725 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/508cf61a06584a56b86fec08da71cf88 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,726 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/58c24e6f62884a5fa818a63cb789ee99 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,726 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/835ce9b34032471f85a7e6927f9b410b for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,726 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/898384d558794c1587adce395ab212a8 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,726 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/dccc9585783b48d9bdfc59dbb6c8ff6f for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,728 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/558a75f660654fa0ac39ae52e8308d6b, top=false 2023-07-12 19:12:05,732 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/44604be4a352490c83f5eba7426442e1, top=false 2023-07-12 19:12:05,732 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/898384d558794c1587adce395ab212a8, top=false 2023-07-12 19:12:05,732 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/a455f6af45424df8b315e7ceaef50dee, top=false 2023-07-12 19:12:05,732 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/62464eabac8747bc8a1535b5d2926158, top=false 2023-07-12 19:12:05,732 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/dccc9585783b48d9bdfc59dbb6c8ff6f, top=false 2023-07-12 19:12:05,733 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/157582c3bdf8489eb50d651833c2fc9d, top=false 2023-07-12 19:12:05,733 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/508cf61a06584a56b86fec08da71cf88, top=false 2023-07-12 19:12:05,734 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/2abe8349bc614a7c8691eb38d8e54b09, top=false 2023-07-12 19:12:05,734 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/5eae4dfef7d54b8887a94366bfadf052, top=false 2023-07-12 19:12:05,738 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/835ce9b34032471f85a7e6927f9b410b, top=false 2023-07-12 19:12:05,740 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/58c24e6f62884a5fa818a63cb789ee99, top=false 2023-07-12 19:12:05,740 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/1afec31ee33a4661bce993525e12afe7, top=false 2023-07-12 19:12:05,744 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/641e38f9379c4a1a923c7a7ff4f4c5b6, top=false 2023-07-12 19:12:05,753 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.splits/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-58c24e6f62884a5fa818a63cb789ee99 for child: 337efae4f3995300de0db431c0f12eeb, parent: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,753 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.splits/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-558a75f660654fa0ac39ae52e8308d6b for child: 337efae4f3995300de0db431c0f12eeb, parent: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,754 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.splits/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-898384d558794c1587adce395ab212a8 for child: 337efae4f3995300de0db431c0f12eeb, parent: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,754 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.splits/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-641e38f9379c4a1a923c7a7ff4f4c5b6 for child: 337efae4f3995300de0db431c0f12eeb, parent: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,754 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.splits/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-835ce9b34032471f85a7e6927f9b410b for child: 337efae4f3995300de0db431c0f12eeb, parent: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,754 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.splits/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-44604be4a352490c83f5eba7426442e1 for child: 337efae4f3995300de0db431c0f12eeb, parent: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,755 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.splits/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-508cf61a06584a56b86fec08da71cf88 for child: 337efae4f3995300de0db431c0f12eeb, parent: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,756 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.splits/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-dccc9585783b48d9bdfc59dbb6c8ff6f for child: 337efae4f3995300de0db431c0f12eeb, parent: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,756 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.splits/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-157582c3bdf8489eb50d651833c2fc9d for child: 337efae4f3995300de0db431c0f12eeb, parent: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,756 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.splits/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-62464eabac8747bc8a1535b5d2926158 for child: 337efae4f3995300de0db431c0f12eeb, parent: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,757 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.splits/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-a455f6af45424df8b315e7ceaef50dee for child: 337efae4f3995300de0db431c0f12eeb, parent: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,757 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.splits/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-5eae4dfef7d54b8887a94366bfadf052 for child: 337efae4f3995300de0db431c0f12eeb, parent: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,758 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/558a75f660654fa0ac39ae52e8308d6b for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,760 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/835ce9b34032471f85a7e6927f9b410b for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,760 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/44604be4a352490c83f5eba7426442e1 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,760 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/58c24e6f62884a5fa818a63cb789ee99 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,761 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/898384d558794c1587adce395ab212a8 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,761 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/508cf61a06584a56b86fec08da71cf88 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,762 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/62464eabac8747bc8a1535b5d2926158 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,762 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/dccc9585783b48d9bdfc59dbb6c8ff6f for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,762 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.splits/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-1afec31ee33a4661bce993525e12afe7 for child: 337efae4f3995300de0db431c0f12eeb, parent: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,763 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/641e38f9379c4a1a923c7a7ff4f4c5b6 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,766 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/a455f6af45424df8b315e7ceaef50dee for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,766 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/157582c3bdf8489eb50d651833c2fc9d for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,766 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/.splits/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-2abe8349bc614a7c8691eb38d8e54b09 for child: 337efae4f3995300de0db431c0f12eeb, parent: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,766 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/5eae4dfef7d54b8887a94366bfadf052 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,766 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/1afec31ee33a4661bce993525e12afe7 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,768 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/2abe8349bc614a7c8691eb38d8e54b09 for region: 891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:05,768 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 891651b57d0223c960e1a67ca99da564 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-07-12 19:12:05,797 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 19:12:05,798 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/7bc70aef82b22e19142528dd3727b1c9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 19:12:05,799 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689189125799"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1689189125799"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1689189125799"}]},"ts":"1689189125799"} 2023-07-12 19:12:05,800 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689189125799"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189125799"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189125799"}]},"ts":"1689189125799"} 2023-07-12 19:12:05,800 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689189125799"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189125799"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189125799"}]},"ts":"1689189125799"} 2023-07-12 19:12:05,802 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=337efae4f3995300de0db431c0f12eeb, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7bc70aef82b22e19142528dd3727b1c9, ASSIGN}] 2023-07-12 19:12:05,803 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7bc70aef82b22e19142528dd3727b1c9, ASSIGN 2023-07-12 19:12:05,803 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7bc70aef82b22e19142528dd3727b1c9, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,33255,1689189020635; forceNewPlan=false, retain=false 2023-07-12 19:12:05,804 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=337efae4f3995300de0db431c0f12eeb, ASSIGN 2023-07-12 19:12:05,804 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=337efae4f3995300de0db431c0f12eeb, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,33255,1689189020635; forceNewPlan=false, retain=false 2023-07-12 19:12:05,954 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-12 19:12:05,955 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=337efae4f3995300de0db431c0f12eeb, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:05,955 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689189125955"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189125955"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189125955"}]},"ts":"1689189125955"} 2023-07-12 19:12:05,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=131, state=RUNNABLE; OpenRegionProcedure 337efae4f3995300de0db431c0f12eeb, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:12:05,959 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=7bc70aef82b22e19142528dd3727b1c9, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:05,959 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689189125959"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189125959"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189125959"}]},"ts":"1689189125959"} 2023-07-12 19:12:05,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=132, state=RUNNABLE; OpenRegionProcedure 7bc70aef82b22e19142528dd3727b1c9, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:12:06,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb. 2023-07-12 19:12:06,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 337efae4f3995300de0db431c0f12eeb, NAME => 'testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.', STARTKEY => '', ENDKEY => '71'} 2023-07-12 19:12:06,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:12:06,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 337efae4f3995300de0db431c0f12eeb 2023-07-12 19:12:06,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:12:06,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 337efae4f3995300de0db431c0f12eeb 2023-07-12 19:12:06,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 337efae4f3995300de0db431c0f12eeb 2023-07-12 19:12:06,115 INFO [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 337efae4f3995300de0db431c0f12eeb 2023-07-12 19:12:06,117 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1 2023-07-12 19:12:06,117 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1 2023-07-12 19:12:06,117 INFO [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 337efae4f3995300de0db431c0f12eeb columnFamilyName cf1 2023-07-12 19:12:06,126 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-1afec31ee33a4661bce993525e12afe7 2023-07-12 19:12:06,130 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-44604be4a352490c83f5eba7426442e1 2023-07-12 19:12:06,134 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-508cf61a06584a56b86fec08da71cf88 2023-07-12 19:12:06,137 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-58c24e6f62884a5fa818a63cb789ee99 2023-07-12 19:12:06,141 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-835ce9b34032471f85a7e6927f9b410b 2023-07-12 19:12:06,144 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-898384d558794c1587adce395ab212a8 2023-07-12 19:12:06,147 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-dccc9585783b48d9bdfc59dbb6c8ff6f 2023-07-12 19:12:06,147 INFO [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(310): Store=337efae4f3995300de0db431c0f12eeb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:06,148 INFO [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 337efae4f3995300de0db431c0f12eeb 2023-07-12 19:12:06,148 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2 2023-07-12 19:12:06,149 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2 2023-07-12 19:12:06,149 INFO [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 337efae4f3995300de0db431c0f12eeb columnFamilyName cf2 2023-07-12 19:12:06,156 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-157582c3bdf8489eb50d651833c2fc9d 2023-07-12 19:12:06,158 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-2abe8349bc614a7c8691eb38d8e54b09 2023-07-12 19:12:06,162 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-558a75f660654fa0ac39ae52e8308d6b 2023-07-12 19:12:06,165 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-5eae4dfef7d54b8887a94366bfadf052 2023-07-12 19:12:06,167 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-62464eabac8747bc8a1535b5d2926158 2023-07-12 19:12:06,170 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-641e38f9379c4a1a923c7a7ff4f4c5b6 2023-07-12 19:12:06,172 DEBUG [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-a455f6af45424df8b315e7ceaef50dee 2023-07-12 19:12:06,173 INFO [StoreOpener-337efae4f3995300de0db431c0f12eeb-1] regionserver.HStore(310): Store=337efae4f3995300de0db431c0f12eeb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:06,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb 2023-07-12 19:12:06,174 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb 2023-07-12 19:12:06,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:12:06,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 337efae4f3995300de0db431c0f12eeb 2023-07-12 19:12:06,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 337efae4f3995300de0db431c0f12eeb; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64573388, jitterRate=-0.03778153657913208}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:12:06,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 337efae4f3995300de0db431c0f12eeb: 2023-07-12 19:12:06,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb., pid=133, masterSystemTime=1689189126108 2023-07-12 19:12:06,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:12:06,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:12:06,179 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:12:06,179 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:12:06,179 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-12 19:12:06,179 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-12 19:12:06,179 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HStore(1912): 337efae4f3995300de0db431c0f12eeb/cf1 is initiating minor compaction (all files) 2023-07-12 19:12:06,179 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HStore(1912): 337efae4f3995300de0db431c0f12eeb/cf2 is initiating minor compaction (all files) 2023-07-12 19:12:06,179 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 337efae4f3995300de0db431c0f12eeb/cf1 in testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb. 2023-07-12 19:12:06,179 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 337efae4f3995300de0db431c0f12eeb/cf2 in testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb. 2023-07-12 19:12:06,180 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-641e38f9379c4a1a923c7a7ff4f4c5b6, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-a455f6af45424df8b315e7ceaef50dee, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-62464eabac8747bc8a1535b5d2926158, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-157582c3bdf8489eb50d651833c2fc9d, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-558a75f660654fa0ac39ae52e8308d6b, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-2abe8349bc614a7c8691eb38d8e54b09, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-5eae4dfef7d54b8887a94366bfadf052] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/.tmp, totalSize=34.9 K 2023-07-12 19:12:06,180 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-58c24e6f62884a5fa818a63cb789ee99, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-835ce9b34032471f85a7e6927f9b410b, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-44604be4a352490c83f5eba7426442e1, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-1afec31ee33a4661bce993525e12afe7, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-dccc9585783b48d9bdfc59dbb6c8ff6f, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-508cf61a06584a56b86fec08da71cf88, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-898384d558794c1587adce395ab212a8] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/.tmp, totalSize=34.9 K 2023-07-12 19:12:06,180 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-641e38f9379c4a1a923c7a7ff4f4c5b6, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689189123924 2023-07-12 19:12:06,180 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-58c24e6f62884a5fa818a63cb789ee99, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689189123924 2023-07-12 19:12:06,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb. 2023-07-12 19:12:06,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb. 2023-07-12 19:12:06,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9. 2023-07-12 19:12:06,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7bc70aef82b22e19142528dd3727b1c9, NAME => 'testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9.', STARTKEY => '71', ENDKEY => ''} 2023-07-12 19:12:06,181 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-a455f6af45424df8b315e7ceaef50dee, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689189124134 2023-07-12 19:12:06,181 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-835ce9b34032471f85a7e6927f9b410b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689189124134 2023-07-12 19:12:06,181 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=337efae4f3995300de0db431c0f12eeb, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:06,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:12:06,181 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689189126181"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189126181"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189126181"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189126181"}]},"ts":"1689189126181"} 2023-07-12 19:12:06,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:06,181 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-44604be4a352490c83f5eba7426442e1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689189124310 2023-07-12 19:12:06,181 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-62464eabac8747bc8a1535b5d2926158, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689189124310 2023-07-12 19:12:06,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:12:06,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:06,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:06,181 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-1afec31ee33a4661bce993525e12afe7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689189124489 2023-07-12 19:12:06,182 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-157582c3bdf8489eb50d651833c2fc9d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689189124489 2023-07-12 19:12:06,182 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-558a75f660654fa0ac39ae52e8308d6b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689189124671 2023-07-12 19:12:06,182 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-dccc9585783b48d9bdfc59dbb6c8ff6f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689189124671 2023-07-12 19:12:06,183 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-2abe8349bc614a7c8691eb38d8e54b09, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689189124860 2023-07-12 19:12:06,183 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-508cf61a06584a56b86fec08da71cf88, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689189124860 2023-07-12 19:12:06,183 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-5eae4dfef7d54b8887a94366bfadf052, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689189125071 2023-07-12 19:12:06,183 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-898384d558794c1587adce395ab212a8, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689189125071 2023-07-12 19:12:06,184 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=131 2023-07-12 19:12:06,184 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=131, state=SUCCESS; OpenRegionProcedure 337efae4f3995300de0db431c0f12eeb, server=jenkins-hbase20.apache.org,33255,1689189020635 in 225 msec 2023-07-12 19:12:06,184 INFO [StoreOpener-7bc70aef82b22e19142528dd3727b1c9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:06,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=337efae4f3995300de0db431c0f12eeb, ASSIGN in 382 msec 2023-07-12 19:12:06,186 DEBUG [StoreOpener-7bc70aef82b22e19142528dd3727b1c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/7bc70aef82b22e19142528dd3727b1c9/cf1 2023-07-12 19:12:06,186 DEBUG [StoreOpener-7bc70aef82b22e19142528dd3727b1c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/7bc70aef82b22e19142528dd3727b1c9/cf1 2023-07-12 19:12:06,186 INFO [StoreOpener-7bc70aef82b22e19142528dd3727b1c9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7bc70aef82b22e19142528dd3727b1c9 columnFamilyName cf1 2023-07-12 19:12:06,187 INFO [StoreOpener-7bc70aef82b22e19142528dd3727b1c9-1] regionserver.HStore(310): Store=7bc70aef82b22e19142528dd3727b1c9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:06,187 INFO [StoreOpener-7bc70aef82b22e19142528dd3727b1c9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:06,188 DEBUG [StoreOpener-7bc70aef82b22e19142528dd3727b1c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/7bc70aef82b22e19142528dd3727b1c9/cf2 2023-07-12 19:12:06,188 DEBUG [StoreOpener-7bc70aef82b22e19142528dd3727b1c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/7bc70aef82b22e19142528dd3727b1c9/cf2 2023-07-12 19:12:06,188 INFO [StoreOpener-7bc70aef82b22e19142528dd3727b1c9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7bc70aef82b22e19142528dd3727b1c9 columnFamilyName cf2 2023-07-12 19:12:06,189 INFO [StoreOpener-7bc70aef82b22e19142528dd3727b1c9-1] regionserver.HStore(310): Store=7bc70aef82b22e19142528dd3727b1c9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:06,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:06,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:06,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:12:06,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:06,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 7bc70aef82b22e19142528dd3727b1c9; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75235498, jitterRate=0.12109628319740295}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:12:06,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 7bc70aef82b22e19142528dd3727b1c9: 2023-07-12 19:12:06,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9., pid=134, masterSystemTime=1689189126108 2023-07-12 19:12:06,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9. 2023-07-12 19:12:06,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9. 2023-07-12 19:12:06,197 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=7bc70aef82b22e19142528dd3727b1c9, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:06,197 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689189126197"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189126197"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189126197"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189126197"}]},"ts":"1689189126197"} 2023-07-12 19:12:06,199 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=132 2023-07-12 19:12:06,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=132, state=SUCCESS; OpenRegionProcedure 7bc70aef82b22e19142528dd3727b1c9, server=jenkins-hbase20.apache.org,33255,1689189020635 in 238 msec 2023-07-12 19:12:06,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=128 2023-07-12 19:12:06,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7bc70aef82b22e19142528dd3727b1c9, ASSIGN in 398 msec 2023-07-12 19:12:06,205 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=891651b57d0223c960e1a67ca99da564, daughterA=337efae4f3995300de0db431c0f12eeb, daughterB=7bc70aef82b22e19142528dd3727b1c9 in 1.1190 sec 2023-07-12 19:12:06,209 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] throttle.PressureAwareThroughputController(145): 337efae4f3995300de0db431c0f12eeb#cf2#compaction#124 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:12:06,209 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] throttle.PressureAwareThroughputController(145): 337efae4f3995300de0db431c0f12eeb#cf1#compaction#125 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:12:06,217 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/.tmp/cf2/f430c702683c49888027a21ce48a96bb as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/f430c702683c49888027a21ce48a96bb 2023-07-12 19:12:06,217 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/.tmp/cf1/f645a42a114c41e1b6f052bb19597345 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/f645a42a114c41e1b6f052bb19597345 2023-07-12 19:12:06,220 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 337efae4f3995300de0db431c0f12eeb/cf1 of 337efae4f3995300de0db431c0f12eeb into f645a42a114c41e1b6f052bb19597345(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:12:06,220 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 337efae4f3995300de0db431c0f12eeb/cf2 of 337efae4f3995300de0db431c0f12eeb into f430c702683c49888027a21ce48a96bb(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:12:06,220 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 337efae4f3995300de0db431c0f12eeb: 2023-07-12 19:12:06,220 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 337efae4f3995300de0db431c0f12eeb: 2023-07-12 19:12:06,220 INFO [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb., storeName=337efae4f3995300de0db431c0f12eeb/cf1, priority=9, startTime=1689189126178; duration=0sec 2023-07-12 19:12:06,220 INFO [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb., storeName=337efae4f3995300de0db431c0f12eeb/cf2, priority=9, startTime=1689189126179; duration=0sec 2023-07-12 19:12:06,221 DEBUG [RS:1;jenkins-hbase20:33255-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:12:06,221 DEBUG [RS:1;jenkins-hbase20:33255-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:12:06,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33255] ipc.CallRunner(144): callId: 989 service: ClientService methodName: Scan size: 157 connection: 148.251.75.209:56298 deadline: 1689189186339, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564. is not online on jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:06,461 DEBUG [Listener at localhost.localdomain/40405] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-07-12 19:12:06,466 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:12:06,467 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-07-12 19:12:06,468 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitTableRegionEmptyDaughter 2023-07-12 19:12:06,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 19:12:06,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-07-12 19:12:06,472 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189126472"}]},"ts":"1689189126472"} 2023-07-12 19:12:06,473 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-07-12 19:12:06,475 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-07-12 19:12:06,475 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=337efae4f3995300de0db431c0f12eeb, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7bc70aef82b22e19142528dd3727b1c9, UNASSIGN}] 2023-07-12 19:12:06,476 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7bc70aef82b22e19142528dd3727b1c9, UNASSIGN 2023-07-12 19:12:06,477 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=7bc70aef82b22e19142528dd3727b1c9, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:06,477 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689189126477"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189126477"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189126477"}]},"ts":"1689189126477"} 2023-07-12 19:12:06,479 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 7bc70aef82b22e19142528dd3727b1c9, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:12:06,481 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=337efae4f3995300de0db431c0f12eeb, UNASSIGN 2023-07-12 19:12:06,482 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=337efae4f3995300de0db431c0f12eeb, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:06,482 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689189126482"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189126482"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189126482"}]},"ts":"1689189126482"} 2023-07-12 19:12:06,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 337efae4f3995300de0db431c0f12eeb, server=jenkins-hbase20.apache.org,33255,1689189020635}] 2023-07-12 19:12:06,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-07-12 19:12:06,631 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:06,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 7bc70aef82b22e19142528dd3727b1c9, disabling compactions & flushes 2023-07-12 19:12:06,632 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9. 2023-07-12 19:12:06,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9. 2023-07-12 19:12:06,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9. after waiting 0 ms 2023-07-12 19:12:06,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9. 2023-07-12 19:12:06,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/7bc70aef82b22e19142528dd3727b1c9/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-07-12 19:12:06,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:12:06,640 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9. 2023-07-12 19:12:06,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 7bc70aef82b22e19142528dd3727b1c9: 2023-07-12 19:12:06,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:06,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 337efae4f3995300de0db431c0f12eeb 2023-07-12 19:12:06,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 337efae4f3995300de0db431c0f12eeb, disabling compactions & flushes 2023-07-12 19:12:06,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb. 2023-07-12 19:12:06,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb. 2023-07-12 19:12:06,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb. after waiting 0 ms 2023-07-12 19:12:06,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb. 2023-07-12 19:12:06,643 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=7bc70aef82b22e19142528dd3727b1c9, regionState=CLOSED 2023-07-12 19:12:06,644 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689189126643"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189126643"}]},"ts":"1689189126643"} 2023-07-12 19:12:06,644 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-58c24e6f62884a5fa818a63cb789ee99, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-835ce9b34032471f85a7e6927f9b410b, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-44604be4a352490c83f5eba7426442e1, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-1afec31ee33a4661bce993525e12afe7, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-dccc9585783b48d9bdfc59dbb6c8ff6f, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-508cf61a06584a56b86fec08da71cf88, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-898384d558794c1587adce395ab212a8] to archive 2023-07-12 19:12:06,645 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:12:06,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-07-12 19:12:06,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 7bc70aef82b22e19142528dd3727b1c9, server=jenkins-hbase20.apache.org,33255,1689189020635 in 167 msec 2023-07-12 19:12:06,648 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-58c24e6f62884a5fa818a63cb789ee99 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-58c24e6f62884a5fa818a63cb789ee99 2023-07-12 19:12:06,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7bc70aef82b22e19142528dd3727b1c9, UNASSIGN in 173 msec 2023-07-12 19:12:06,650 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-835ce9b34032471f85a7e6927f9b410b to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-835ce9b34032471f85a7e6927f9b410b 2023-07-12 19:12:06,652 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-44604be4a352490c83f5eba7426442e1 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-44604be4a352490c83f5eba7426442e1 2023-07-12 19:12:06,653 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-1afec31ee33a4661bce993525e12afe7 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-1afec31ee33a4661bce993525e12afe7 2023-07-12 19:12:06,654 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-dccc9585783b48d9bdfc59dbb6c8ff6f to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-dccc9585783b48d9bdfc59dbb6c8ff6f 2023-07-12 19:12:06,656 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-508cf61a06584a56b86fec08da71cf88 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-508cf61a06584a56b86fec08da71cf88 2023-07-12 19:12:06,656 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-898384d558794c1587adce395ab212a8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-898384d558794c1587adce395ab212a8 2023-07-12 19:12:06,658 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-641e38f9379c4a1a923c7a7ff4f4c5b6, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-a455f6af45424df8b315e7ceaef50dee, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-62464eabac8747bc8a1535b5d2926158, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-157582c3bdf8489eb50d651833c2fc9d, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-558a75f660654fa0ac39ae52e8308d6b, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-2abe8349bc614a7c8691eb38d8e54b09, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-5eae4dfef7d54b8887a94366bfadf052] to archive 2023-07-12 19:12:06,658 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:12:06,659 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-641e38f9379c4a1a923c7a7ff4f4c5b6 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-641e38f9379c4a1a923c7a7ff4f4c5b6 2023-07-12 19:12:06,660 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-a455f6af45424df8b315e7ceaef50dee to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-a455f6af45424df8b315e7ceaef50dee 2023-07-12 19:12:06,661 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-62464eabac8747bc8a1535b5d2926158 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-62464eabac8747bc8a1535b5d2926158 2023-07-12 19:12:06,661 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-157582c3bdf8489eb50d651833c2fc9d to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-157582c3bdf8489eb50d651833c2fc9d 2023-07-12 19:12:06,662 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-558a75f660654fa0ac39ae52e8308d6b to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-558a75f660654fa0ac39ae52e8308d6b 2023-07-12 19:12:06,663 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-2abe8349bc614a7c8691eb38d8e54b09 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-2abe8349bc614a7c8691eb38d8e54b09 2023-07-12 19:12:06,663 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-5eae4dfef7d54b8887a94366bfadf052 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/testSplitTableRegionEmptyDaughter=891651b57d0223c960e1a67ca99da564-5eae4dfef7d54b8887a94366bfadf052 2023-07-12 19:12:06,666 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 19:12:06,666 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:12:06,666 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb. 2023-07-12 19:12:06,666 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 337efae4f3995300de0db431c0f12eeb: 2023-07-12 19:12:06,668 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 337efae4f3995300de0db431c0f12eeb 2023-07-12 19:12:06,668 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=337efae4f3995300de0db431c0f12eeb, regionState=CLOSED 2023-07-12 19:12:06,668 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689189126668"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189126668"}]},"ts":"1689189126668"} 2023-07-12 19:12:06,670 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-07-12 19:12:06,671 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 337efae4f3995300de0db431c0f12eeb, server=jenkins-hbase20.apache.org,33255,1689189020635 in 186 msec 2023-07-12 19:12:06,672 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-07-12 19:12:06,672 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=337efae4f3995300de0db431c0f12eeb, UNASSIGN in 196 msec 2023-07-12 19:12:06,673 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189126673"}]},"ts":"1689189126673"} 2023-07-12 19:12:06,674 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-07-12 19:12:06,675 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-07-12 19:12:06,676 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 207 msec 2023-07-12 19:12:06,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-07-12 19:12:06,776 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-07-12 19:12:06,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitTableRegionEmptyDaughter 2023-07-12 19:12:06,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 19:12:06,783 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 19:12:06,784 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 19:12:06,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-07-12 19:12:06,791 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:06,791 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:06,791 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb 2023-07-12 19:12:06,794 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7bc70aef82b22e19142528dd3727b1c9/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7bc70aef82b22e19142528dd3727b1c9/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7bc70aef82b22e19142528dd3727b1c9/recovered.edits] 2023-07-12 19:12:06,794 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/recovered.edits] 2023-07-12 19:12:06,794 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/recovered.edits] 2023-07-12 19:12:06,796 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/f645a42a114c41e1b6f052bb19597345 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf1/f645a42a114c41e1b6f052bb19597345 2023-07-12 19:12:06,796 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/1afec31ee33a4661bce993525e12afe7 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/1afec31ee33a4661bce993525e12afe7 2023-07-12 19:12:06,797 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/44604be4a352490c83f5eba7426442e1 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/44604be4a352490c83f5eba7426442e1 2023-07-12 19:12:06,798 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7bc70aef82b22e19142528dd3727b1c9/recovered.edits/88.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/7bc70aef82b22e19142528dd3727b1c9/recovered.edits/88.seqid 2023-07-12 19:12:06,798 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/f430c702683c49888027a21ce48a96bb to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/cf2/f430c702683c49888027a21ce48a96bb 2023-07-12 19:12:06,798 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/508cf61a06584a56b86fec08da71cf88 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/508cf61a06584a56b86fec08da71cf88 2023-07-12 19:12:06,798 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7bc70aef82b22e19142528dd3727b1c9 2023-07-12 19:12:06,799 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/58c24e6f62884a5fa818a63cb789ee99 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/58c24e6f62884a5fa818a63cb789ee99 2023-07-12 19:12:06,800 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/recovered.edits/90.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb/recovered.edits/90.seqid 2023-07-12 19:12:06,800 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/835ce9b34032471f85a7e6927f9b410b to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/835ce9b34032471f85a7e6927f9b410b 2023-07-12 19:12:06,800 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/337efae4f3995300de0db431c0f12eeb 2023-07-12 19:12:06,800 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/898384d558794c1587adce395ab212a8 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/898384d558794c1587adce395ab212a8 2023-07-12 19:12:06,801 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/dccc9585783b48d9bdfc59dbb6c8ff6f to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf1/dccc9585783b48d9bdfc59dbb6c8ff6f 2023-07-12 19:12:06,803 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/157582c3bdf8489eb50d651833c2fc9d to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/157582c3bdf8489eb50d651833c2fc9d 2023-07-12 19:12:06,804 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/2abe8349bc614a7c8691eb38d8e54b09 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/2abe8349bc614a7c8691eb38d8e54b09 2023-07-12 19:12:06,805 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/558a75f660654fa0ac39ae52e8308d6b to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/558a75f660654fa0ac39ae52e8308d6b 2023-07-12 19:12:06,805 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/5eae4dfef7d54b8887a94366bfadf052 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/5eae4dfef7d54b8887a94366bfadf052 2023-07-12 19:12:06,806 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/62464eabac8747bc8a1535b5d2926158 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/62464eabac8747bc8a1535b5d2926158 2023-07-12 19:12:06,807 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/641e38f9379c4a1a923c7a7ff4f4c5b6 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/641e38f9379c4a1a923c7a7ff4f4c5b6 2023-07-12 19:12:06,808 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/a455f6af45424df8b315e7ceaef50dee to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/cf2/a455f6af45424df8b315e7ceaef50dee 2023-07-12 19:12:06,810 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/recovered.edits/85.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564/recovered.edits/85.seqid 2023-07-12 19:12:06,811 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegionEmptyDaughter/891651b57d0223c960e1a67ca99da564 2023-07-12 19:12:06,811 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-07-12 19:12:06,813 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 19:12:06,815 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-07-12 19:12:06,816 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-07-12 19:12:06,817 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 19:12:06,817 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-07-12 19:12:06,818 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189126818"}]},"ts":"9223372036854775807"} 2023-07-12 19:12:06,818 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189126818"}]},"ts":"9223372036854775807"} 2023-07-12 19:12:06,818 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189126818"}]},"ts":"9223372036854775807"} 2023-07-12 19:12:06,819 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-12 19:12:06,819 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 891651b57d0223c960e1a67ca99da564, NAME => 'testSplitTableRegionEmptyDaughter,,1689189123417.891651b57d0223c960e1a67ca99da564.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 337efae4f3995300de0db431c0f12eeb, NAME => 'testSplitTableRegionEmptyDaughter,,1689189125085.337efae4f3995300de0db431c0f12eeb.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 7bc70aef82b22e19142528dd3727b1c9, NAME => 'testSplitTableRegionEmptyDaughter,71,1689189125085.7bc70aef82b22e19142528dd3727b1c9.', STARTKEY => '71', ENDKEY => ''}] 2023-07-12 19:12:06,819 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-07-12 19:12:06,819 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689189126819"}]},"ts":"9223372036854775807"} 2023-07-12 19:12:06,821 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-07-12 19:12:06,822 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 19:12:06,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 44 msec 2023-07-12 19:12:06,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-07-12 19:12:06,887 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-07-12 19:12:06,905 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375 (was 376), OpenFileDescriptor=833 (was 835), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=85 (was 93), ProcessCount=170 (was 170), AvailableMemoryMB=5728 (was 5735) 2023-07-12 19:12:06,921 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=375, OpenFileDescriptor=833, MaxFileDescriptor=60000, SystemLoadAverage=85, ProcessCount=170, AvailableMemoryMB=5727 2023-07-12 19:12:06,921 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:12:06,923 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 19:12:06,923 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 19:12:06,924 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-07-12 19:12:06,926 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 19:12:06,926 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 19:12:06,927 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:06,928 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60 empty. 2023-07-12 19:12:06,928 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:06,928 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-07-12 19:12:06,940 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-07-12 19:12:06,941 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => f9f0786a2f56ee8f585bc0442b614e60, NAME => 'testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp 2023-07-12 19:12:06,948 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:12:06,948 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1604): Closing f9f0786a2f56ee8f585bc0442b614e60, disabling compactions & flushes 2023-07-12 19:12:06,948 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:06,948 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:06,948 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. after waiting 0 ms 2023-07-12 19:12:06,948 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:06,948 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:06,948 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1558): Region close journal for f9f0786a2f56ee8f585bc0442b614e60: 2023-07-12 19:12:06,950 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 19:12:06,951 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689189126951"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189126951"}]},"ts":"1689189126951"} 2023-07-12 19:12:06,952 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 19:12:06,953 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 19:12:06,953 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189126953"}]},"ts":"1689189126953"} 2023-07-12 19:12:06,954 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-07-12 19:12:06,955 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 19:12:06,956 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 19:12:06,956 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 19:12:06,956 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 19:12:06,956 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 19:12:06,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=f9f0786a2f56ee8f585bc0442b614e60, ASSIGN}] 2023-07-12 19:12:06,958 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=f9f0786a2f56ee8f585bc0442b614e60, ASSIGN 2023-07-12 19:12:06,958 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=f9f0786a2f56ee8f585bc0442b614e60, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,38353,1689189020667; forceNewPlan=false, retain=false 2023-07-12 19:12:07,108 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 19:12:07,111 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=f9f0786a2f56ee8f585bc0442b614e60, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,111 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689189127111"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189127111"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189127111"}]},"ts":"1689189127111"} 2023-07-12 19:12:07,115 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure f9f0786a2f56ee8f585bc0442b614e60, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:12:07,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:07,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f9f0786a2f56ee8f585bc0442b614e60, NAME => 'testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60.', STARTKEY => '', ENDKEY => ''} 2023-07-12 19:12:07,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:12:07,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:07,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:12:07,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:07,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:07,283 INFO [StoreOpener-f9f0786a2f56ee8f585bc0442b614e60-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:07,285 DEBUG [StoreOpener-f9f0786a2f56ee8f585bc0442b614e60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1 2023-07-12 19:12:07,285 DEBUG [StoreOpener-f9f0786a2f56ee8f585bc0442b614e60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1 2023-07-12 19:12:07,285 INFO [StoreOpener-f9f0786a2f56ee8f585bc0442b614e60-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f9f0786a2f56ee8f585bc0442b614e60 columnFamilyName cf1 2023-07-12 19:12:07,286 INFO [StoreOpener-f9f0786a2f56ee8f585bc0442b614e60-1] regionserver.HStore(310): Store=f9f0786a2f56ee8f585bc0442b614e60/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:07,286 INFO [StoreOpener-f9f0786a2f56ee8f585bc0442b614e60-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:07,287 DEBUG [StoreOpener-f9f0786a2f56ee8f585bc0442b614e60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2 2023-07-12 19:12:07,287 DEBUG [StoreOpener-f9f0786a2f56ee8f585bc0442b614e60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2 2023-07-12 19:12:07,288 INFO [StoreOpener-f9f0786a2f56ee8f585bc0442b614e60-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f9f0786a2f56ee8f585bc0442b614e60 columnFamilyName cf2 2023-07-12 19:12:07,288 INFO [StoreOpener-f9f0786a2f56ee8f585bc0442b614e60-1] regionserver.HStore(310): Store=f9f0786a2f56ee8f585bc0442b614e60/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:07,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:07,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:07,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:12:07,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:07,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 19:12:07,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened f9f0786a2f56ee8f585bc0442b614e60; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62753044, jitterRate=-0.0649067759513855}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:12:07,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for f9f0786a2f56ee8f585bc0442b614e60: 2023-07-12 19:12:07,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60., pid=143, masterSystemTime=1689189127268 2023-07-12 19:12:07,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:07,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:07,298 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=f9f0786a2f56ee8f585bc0442b614e60, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,298 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689189127298"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189127298"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189127298"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189127298"}]},"ts":"1689189127298"} 2023-07-12 19:12:07,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-07-12 19:12:07,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure f9f0786a2f56ee8f585bc0442b614e60, server=jenkins-hbase20.apache.org,38353,1689189020667 in 184 msec 2023-07-12 19:12:07,302 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-07-12 19:12:07,302 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=f9f0786a2f56ee8f585bc0442b614e60, ASSIGN in 344 msec 2023-07-12 19:12:07,303 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 19:12:07,303 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189127303"}]},"ts":"1689189127303"} 2023-07-12 19:12:07,304 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-07-12 19:12:07,305 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 19:12:07,307 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 382 msec 2023-07-12 19:12:07,442 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:12:07,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-12 19:12:07,445 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-12 19:12:07,445 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:07,445 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-12 19:12:07,445 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-12 19:12:07,446 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,446 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,447 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,447 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,447 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,447 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,448 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,449 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:12:07,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,449 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,449 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,450 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:07,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:07,450 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:07,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:07,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,451 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:07,451 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:07,451 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:07,451 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:07,451 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:07,451 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:07,451 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:07,451 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:07,451 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:07,451 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:07,451 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:07,452 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:07,452 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:07,452 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:07,452 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:07,452 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. started... 2023-07-12 19:12:07,452 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:07,452 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:07,454 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:07,452 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:12:07,454 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:12:07,453 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9f0786a2f56ee8f585bc0442b614e60 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 19:12:07,453 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:07,455 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,455 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,456 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,456 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:07,456 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,456 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:07,466 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf1/46a6c3391d354a599ca7cd58973a4fda 2023-07-12 19:12:07,480 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf2/3c36f779a1ba409f95aace46e1091f93 2023-07-12 19:12:07,484 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf1/46a6c3391d354a599ca7cd58973a4fda as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/46a6c3391d354a599ca7cd58973a4fda 2023-07-12 19:12:07,487 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/46a6c3391d354a599ca7cd58973a4fda, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:12:07,487 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf2/3c36f779a1ba409f95aace46e1091f93 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/3c36f779a1ba409f95aace46e1091f93 2023-07-12 19:12:07,491 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/3c36f779a1ba409f95aace46e1091f93, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:12:07,491 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for f9f0786a2f56ee8f585bc0442b614e60 in 38ms, sequenceid=6, compaction requested=false 2023-07-12 19:12:07,491 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-07-12 19:12:07,492 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9f0786a2f56ee8f585bc0442b614e60: 2023-07-12 19:12:07,492 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:07,492 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:12:07,492 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:12:07,492 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,492 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:07,492 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:07,493 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,493 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:07,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:07,493 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,493 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:07,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:07,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:07,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:07,495 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-12 19:12:07,495 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56f75ddc[Count = 0] remaining members to acquire global barrier 2023-07-12 19:12:07,495 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-12 19:12:07,495 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,497 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,497 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,497 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,497 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:07,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,497 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:07,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,497 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:12:07,497 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:07,497 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:07,497 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:12:07,497 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,497 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:12:07,497 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:07,497 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:07,497 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:12:07,498 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,498 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:07,498 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:07,498 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:07,498 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:07,498 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:07,498 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:07,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:07,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:07,498 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:07,498 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:07,498 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:07,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:07,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:07,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:07,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:12:07,501 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-12 19:12:07,501 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-12 19:12:07,501 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:12:07,501 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:12:07,501 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-12 19:12:07,501 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:12:07,502 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,502 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,502 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,502 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,502 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,502 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,502 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,502 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:07,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:07,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,503 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:07,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,503 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:07,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,504 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:07,504 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,505 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,506 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,506 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,506 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,506 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,506 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,506 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,506 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:07,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,506 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,506 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,507 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-12 19:12:07,507 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:12:07,507 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,506 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,507 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-12 19:12:07,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,507 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,507 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:12:07,507 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,507 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,507 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,507 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,507 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,507 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,607 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:12:07,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:12:07,622 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:12:07,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-12 19:12:07,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-12 19:12:07,624 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-12 19:12:07,624 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:07,624 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-12 19:12:07,624 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-12 19:12:07,625 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,625 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,626 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,626 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,626 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,626 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,626 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,626 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:12:07,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,627 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:07,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,627 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,627 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:07,627 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,628 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 19:12:07,628 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:07,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:07,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:07,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,628 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:07,628 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:07,628 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:07,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,628 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:07,628 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:07,628 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:07,628 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:07,628 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 19:12:07,628 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:07,629 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 19:12:07,629 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:07,629 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:07,629 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:07,629 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:07,629 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:07,629 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:07,629 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:07,629 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:07,629 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:07,629 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:12:07,629 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:12:07,629 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:07,629 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. started... 2023-07-12 19:12:07,629 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,629 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9f0786a2f56ee8f585bc0442b614e60 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:12:07,630 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,630 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:07,630 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,631 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,631 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:07,640 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf1/0be45eb572614608a7356cb313a7295f 2023-07-12 19:12:07,651 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf2/110063a072624e989128d70308f4d383 2023-07-12 19:12:07,654 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf1/0be45eb572614608a7356cb313a7295f as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/0be45eb572614608a7356cb313a7295f 2023-07-12 19:12:07,658 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/0be45eb572614608a7356cb313a7295f, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:12:07,658 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf2/110063a072624e989128d70308f4d383 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/110063a072624e989128d70308f4d383 2023-07-12 19:12:07,661 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/110063a072624e989128d70308f4d383, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 19:12:07,662 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f9f0786a2f56ee8f585bc0442b614e60 in 33ms, sequenceid=19, compaction requested=false 2023-07-12 19:12:07,662 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9f0786a2f56ee8f585bc0442b614e60: 2023-07-12 19:12:07,662 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:07,662 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:12:07,662 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:12:07,662 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,662 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:07,662 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:07,663 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,663 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:07,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:07,663 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,663 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:07,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:07,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:07,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:07,665 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-12 19:12:07,665 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@384712ac[Count = 0] remaining members to acquire global barrier 2023-07-12 19:12:07,665 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-12 19:12:07,665 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,666 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,666 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,666 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,666 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:07,666 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:07,666 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,666 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:12:07,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,666 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:12:07,666 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:07,666 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:07,666 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:12:07,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,667 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:07,667 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:07,667 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:12:07,667 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,667 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:07,667 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:07,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:07,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:07,667 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:07,667 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:07,667 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:07,667 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:07,667 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:07,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:07,668 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:07,668 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:07,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:07,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:07,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:12:07,670 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-12 19:12:07,670 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-12 19:12:07,670 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:12:07,670 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:12:07,670 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-12 19:12:07,670 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:12:07,671 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,671 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,671 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:07,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:07,671 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,671 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,671 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:07,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,672 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,672 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,672 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:07,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,673 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:07,674 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,675 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,675 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,678 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,678 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,678 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,678 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,678 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,678 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,678 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,679 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:07,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,678 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,679 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-12 19:12:07,679 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:12:07,679 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,679 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,679 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,679 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-12 19:12:07,679 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,680 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,680 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:12:07,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,680 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,680 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,680 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,680 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,780 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:12:07,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:12:07,800 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:12:07,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-12 19:12:07,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-12 19:12:07,803 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-12 19:12:07,803 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:07,803 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-12 19:12:07,803 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-12 19:12:07,803 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,803 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,804 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,804 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,804 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,804 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,805 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,805 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:12:07,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,806 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:07,806 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:07,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,806 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:07,806 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:07,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:07,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,806 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:07,806 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 19:12:07,807 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:07,807 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 19:12:07,807 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:07,807 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:07,807 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 19:12:07,807 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:07,807 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:07,807 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:07,807 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:07,807 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:07,807 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:07,807 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:07,807 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:07,807 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:07,807 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:07,807 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:12:07,807 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:12:07,807 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:07,807 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. started... 2023-07-12 19:12:07,807 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:07,807 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:07,807 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9f0786a2f56ee8f585bc0442b614e60 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:12:07,807 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:07,808 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,808 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,808 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,808 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:07,809 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,809 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:07,821 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf1/9906a68f49fe4e3da5382d828bb03e20 2023-07-12 19:12:07,832 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf2/f1ccf904c39e41fcbfa8ed444294f08d 2023-07-12 19:12:07,835 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf1/9906a68f49fe4e3da5382d828bb03e20 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/9906a68f49fe4e3da5382d828bb03e20 2023-07-12 19:12:07,838 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/9906a68f49fe4e3da5382d828bb03e20, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:12:07,839 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf2/f1ccf904c39e41fcbfa8ed444294f08d as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/f1ccf904c39e41fcbfa8ed444294f08d 2023-07-12 19:12:07,842 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/f1ccf904c39e41fcbfa8ed444294f08d, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 19:12:07,843 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f9f0786a2f56ee8f585bc0442b614e60 in 36ms, sequenceid=32, compaction requested=true 2023-07-12 19:12:07,843 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9f0786a2f56ee8f585bc0442b614e60: 2023-07-12 19:12:07,843 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:07,843 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:12:07,844 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:12:07,844 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,844 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:07,844 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:07,845 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:07,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:07,845 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:07,845 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,845 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:07,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:07,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:07,847 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-12 19:12:07,847 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5638832b[Count = 0] remaining members to acquire global barrier 2023-07-12 19:12:07,847 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-12 19:12:07,847 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,848 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,848 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,848 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,848 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:07,848 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,848 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:12:07,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,848 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:07,848 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:07,848 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:07,848 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:12:07,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,848 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:12:07,848 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:07,848 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:07,848 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:12:07,849 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:07,849 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:07,849 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:07,849 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,849 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:07,849 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:07,849 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:07,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:07,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:07,849 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:07,849 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:07,850 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:07,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:07,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:07,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:07,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:12:07,853 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-12 19:12:07,853 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-12 19:12:07,853 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:12:07,853 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:12:07,853 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-12 19:12:07,853 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:12:07,854 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,854 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,854 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,854 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,854 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,854 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:07,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:07,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,855 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,854 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:07,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,855 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,856 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:07,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,857 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:07,858 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:07,858 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,863 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,863 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,863 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,863 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:07,863 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:07,863 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-12 19:12:07,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:12:07,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,863 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,863 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:07,863 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:07,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:07,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,863 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-12 19:12:07,863 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:07,864 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,864 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:12:07,864 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:07,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:07,864 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:07,864 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,864 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:07,864 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:07,864 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:07,864 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:07,964 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:12:07,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:12:08,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:12:08,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-12 19:12:08,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-12 19:12:08,120 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-12 19:12:08,120 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:08,120 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-12 19:12:08,120 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-12 19:12:08,120 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,120 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,121 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,121 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,121 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,121 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,121 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,122 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:12:08,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,122 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,122 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:08,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:08,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,122 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:08,122 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,122 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:08,123 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:08,123 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:08,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:08,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,123 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:08,123 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:08,123 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:08,123 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:08,123 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:08,123 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:08,123 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:08,123 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:08,123 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:08,123 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:08,123 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:08,123 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:08,123 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:08,123 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:08,123 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:08,123 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:12:08,123 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:12:08,123 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:08,123 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. started... 2023-07-12 19:12:08,124 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9f0786a2f56ee8f585bc0442b614e60 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:12:08,124 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,124 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,124 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,124 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:08,124 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,124 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:08,133 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf1/e36367a5685b4629984cb2521a64739e 2023-07-12 19:12:08,144 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf2/1b04c15cecd5423a9e295067eff9a5ec 2023-07-12 19:12:08,147 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf1/e36367a5685b4629984cb2521a64739e as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/e36367a5685b4629984cb2521a64739e 2023-07-12 19:12:08,150 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/e36367a5685b4629984cb2521a64739e, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:12:08,151 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf2/1b04c15cecd5423a9e295067eff9a5ec as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/1b04c15cecd5423a9e295067eff9a5ec 2023-07-12 19:12:08,153 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/1b04c15cecd5423a9e295067eff9a5ec, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 19:12:08,154 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f9f0786a2f56ee8f585bc0442b614e60 in 30ms, sequenceid=45, compaction requested=true 2023-07-12 19:12:08,154 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9f0786a2f56ee8f585bc0442b614e60: 2023-07-12 19:12:08,154 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:08,154 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:12:08,154 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:12:08,154 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,154 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:08,154 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:08,155 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,155 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:08,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:08,155 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,155 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:08,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:08,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:08,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:08,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-12 19:12:08,157 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-12 19:12:08,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79d9054d[Count = 0] remaining members to acquire global barrier 2023-07-12 19:12:08,157 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,157 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,157 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,157 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,157 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:08,158 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:08,157 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:08,158 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:08,158 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:12:08,157 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,158 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:12:08,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,158 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:12:08,158 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:08,158 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:08,158 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:12:08,158 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:08,158 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:08,158 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:08,158 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:08,158 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:08,158 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:08,159 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,159 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:08,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:08,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:08,159 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:08,160 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:08,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:08,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:08,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:08,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:12:08,163 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-12 19:12:08,163 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-12 19:12:08,163 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:12:08,163 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:12:08,163 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-12 19:12:08,163 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:12:08,164 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,164 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,164 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:08,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:08,164 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,164 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,164 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:08,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,164 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,164 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:08,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,165 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,166 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:08,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,167 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,168 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,168 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,170 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,170 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,170 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,170 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,170 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,170 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,170 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,170 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,170 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,170 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-12 19:12:08,170 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:12:08,170 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,170 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,170 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:08,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,170 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-12 19:12:08,170 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,171 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:12:08,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,170 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,171 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,171 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,171 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,171 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,271 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:12:08,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:12:08,293 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:12:08,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-12 19:12:08,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-12 19:12:08,294 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-12 19:12:08,295 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:08,295 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-12 19:12:08,295 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-12 19:12:08,296 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,296 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,297 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,297 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,297 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,297 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,297 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,298 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:12:08,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,299 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,299 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,299 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:08,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:08,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,299 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:08,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:08,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,300 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:08,300 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 19:12:08,299 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:08,300 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 19:12:08,300 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:08,300 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:08,300 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:08,300 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:08,300 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:08,300 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 19:12:08,300 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:08,300 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:12:08,301 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:12:08,300 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:08,300 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:08,301 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:08,301 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:08,301 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:08,301 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:08,301 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:08,301 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:08,301 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. started... 2023-07-12 19:12:08,301 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:08,301 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:08,301 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:08,301 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9f0786a2f56ee8f585bc0442b614e60 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:12:08,304 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,305 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,305 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,305 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:08,305 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,305 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:08,315 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0 2023-07-12 19:12:08,326 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf2/a18008abe86b4fabb316a7f979e75f2c 2023-07-12 19:12:08,330 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0 2023-07-12 19:12:08,334 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:12:08,334 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf2/a18008abe86b4fabb316a7f979e75f2c as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/a18008abe86b4fabb316a7f979e75f2c 2023-07-12 19:12:08,338 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/a18008abe86b4fabb316a7f979e75f2c, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 19:12:08,339 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f9f0786a2f56ee8f585bc0442b614e60 in 38ms, sequenceid=58, compaction requested=true 2023-07-12 19:12:08,339 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9f0786a2f56ee8f585bc0442b614e60: 2023-07-12 19:12:08,339 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:08,339 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:12:08,339 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:12:08,339 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,339 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:08,339 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:08,340 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,341 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:08,341 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,341 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:08,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:08,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-12 19:12:08,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74de433[Count = 0] remaining members to acquire global barrier 2023-07-12 19:12:08,343 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-12 19:12:08,344 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,344 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,344 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,344 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,345 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,345 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:12:08,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,345 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:08,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,345 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:08,345 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:08,345 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:08,345 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:12:08,345 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:08,345 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:08,345 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:12:08,345 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:12:08,346 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:08,346 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:08,346 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,346 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:08,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,346 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:08,346 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:08,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:08,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:08,346 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:08,347 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:08,347 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:08,347 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:08,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:08,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:08,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:08,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:12:08,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-12 19:12:08,351 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-12 19:12:08,351 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:12:08,351 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:12:08,351 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-12 19:12:08,351 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:12:08,354 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,354 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:08,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:08,354 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,354 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,355 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:08,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,354 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,355 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,355 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,356 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:08,357 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:08,360 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,360 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,361 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,364 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,364 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,364 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,364 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,364 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,364 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:08,364 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,364 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,364 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,364 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-12 19:12:08,364 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:12:08,364 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,364 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,364 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,365 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-12 19:12:08,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,365 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:12:08,365 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,365 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,365 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,365 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,365 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,365 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,465 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:12:08,466 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:12:08,476 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 19:12:08,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-12 19:12:08,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-12 19:12:08,477 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-12 19:12:08,477 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:08,477 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-12 19:12:08,477 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-12 19:12:08,477 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,477 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,478 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,478 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,478 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,478 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,478 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,479 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 19:12:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,479 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,479 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,479 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,479 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,479 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:08,479 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 19:12:08,479 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 19:12:08,479 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:08,479 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 19:12:08,480 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:08,480 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:08,479 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:08,480 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:08,480 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:08,480 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33255,1689189020635' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,480 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:08,480 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 19:12:08,480 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:08,480 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 19:12:08,480 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 19:12:08,480 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 19:12:08,480 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 19:12:08,480 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:08,480 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 19:12:08,480 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 19:12:08,480 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. started... 2023-07-12 19:12:08,480 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 19:12:08,480 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:08,480 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,45655,1689189020596' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:08,480 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9f0786a2f56ee8f585bc0442b614e60 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 19:12:08,481 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,484 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,485 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,485 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:08,485 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,485 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:08,493 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf1/ea2e70b6c6934af9b3859991c789e08d 2023-07-12 19:12:08,501 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf2/43a2d20956464bad87f3936525e2e4e0 2023-07-12 19:12:08,505 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf1/ea2e70b6c6934af9b3859991c789e08d as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ea2e70b6c6934af9b3859991c789e08d 2023-07-12 19:12:08,508 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ea2e70b6c6934af9b3859991c789e08d, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:12:08,509 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf2/43a2d20956464bad87f3936525e2e4e0 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/43a2d20956464bad87f3936525e2e4e0 2023-07-12 19:12:08,512 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/43a2d20956464bad87f3936525e2e4e0, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 19:12:08,513 INFO [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f9f0786a2f56ee8f585bc0442b614e60 in 33ms, sequenceid=71, compaction requested=true 2023-07-12 19:12:08,513 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9f0786a2f56ee8f585bc0442b614e60: 2023-07-12 19:12:08,513 DEBUG [rs(jenkins-hbase20.apache.org,38353,1689189020667)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:08,513 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 19:12:08,513 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 19:12:08,513 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,513 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 19:12:08,513 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 19:12:08,514 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,514 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:08,514 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,514 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 19:12:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:08,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,38353,1689189020667' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-12 19:12:08,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@360f7afa[Count = 0] remaining members to acquire global barrier 2023-07-12 19:12:08,515 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-12 19:12:08,515 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,516 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,516 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,516 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,516 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,516 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 19:12:08,516 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:08,516 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:08,516 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:08,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,516 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,45655,1689189020596' in zk 2023-07-12 19:12:08,516 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:08,516 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33255,1689189020635' in zk 2023-07-12 19:12:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,516 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 19:12:08,516 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 19:12:08,516 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,38353,1689189020667' in zk 2023-07-12 19:12:08,517 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:08,517 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:08,517 DEBUG [member: 'jenkins-hbase20.apache.org,45655,1689189020596' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:08,517 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:08,517 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 19:12:08,517 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:08,517 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,517 DEBUG [member: 'jenkins-hbase20.apache.org,33255,1689189020635' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:08,517 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:08,517 DEBUG [member: 'jenkins-hbase20.apache.org,38353,1689189020667' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 19:12:08,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:08,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:08,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:08,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,38353,1689189020667': 2023-07-12 19:12:08,520 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,38353,1689189020667' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-12 19:12:08,520 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-12 19:12:08,520 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 19:12:08,520 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 19:12:08,520 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-12 19:12:08,520 INFO [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 19:12:08,521 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,521 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,521 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,521 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,521 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,521 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,521 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,521 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,521 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 19:12:08,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 19:12:08,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,522 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 19:12:08,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 19:12:08,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,523 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,523 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,523 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 19:12:08,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 19:12:08,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,525 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,525 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,525 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,525 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 19:12:08,525 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,525 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,525 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-12 19:12:08,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,525 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,525 DEBUG [(jenkins-hbase20.apache.org,43565,1689189019651)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 19:12:08,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 19:12:08,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 19:12:08,525 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 19:12:08,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,526 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 19:12:08,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 19:12:08,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,526 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 19:12:08,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 19:12:08,526 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:08,526 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,526 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-12 19:12:08,526 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:08,526 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 19:12:08,526 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,526 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 19:12:08,526 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 19:12:08,626 ERROR [BP-1020377755-148.251.75.209-1689189016963 heartbeating to localhost.localdomain/127.0.0.1:43855] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1020377755-148.251.75.209-1689189016963 (Datanode Uuid 9225ce29-b06f-40b1-aaea-24784a80c6f4) service to localhost.localdomain/127.0.0.1:43855 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-12 19:12:08,626 DEBUG [Listener at localhost.localdomain/40405] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 19:12:08,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 19:12:08,634 DEBUG [Listener at localhost.localdomain/40405] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,634 DEBUG [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=f9f0786a2f56ee8f585bc0442b614e60, daughterA=73093ec69a7f12af7e4d92143adcac5d, daughterB=45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:08,635 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=f9f0786a2f56ee8f585bc0442b614e60, daughterA=73093ec69a7f12af7e4d92143adcac5d, daughterB=45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:08,635 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=f9f0786a2f56ee8f585bc0442b614e60, daughterA=73093ec69a7f12af7e4d92143adcac5d, daughterB=45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:08,636 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=f9f0786a2f56ee8f585bc0442b614e60, daughterA=73093ec69a7f12af7e4d92143adcac5d, daughterB=45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:08,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=f9f0786a2f56ee8f585bc0442b614e60, UNASSIGN}] 2023-07-12 19:12:08,638 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=f9f0786a2f56ee8f585bc0442b614e60, UNASSIGN 2023-07-12 19:12:08,638 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=f9f0786a2f56ee8f585bc0442b614e60, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:08,638 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689189128638"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189128638"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189128638"}]},"ts":"1689189128638"} 2023-07-12 19:12:08,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure f9f0786a2f56ee8f585bc0442b614e60, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:12:08,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing f9f0786a2f56ee8f585bc0442b614e60, disabling compactions & flushes 2023-07-12 19:12:08,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:08,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:08,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. after waiting 0 ms 2023-07-12 19:12:08,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:08,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing f9f0786a2f56ee8f585bc0442b614e60 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-12 19:12:08,798 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf1/ab9e93dae739448d81c79165f0d0a1c3 2023-07-12 19:12:08,807 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf2/8014c3f80ce94dccafdd8d11634fbcb3 2023-07-12 19:12:08,811 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf1/ab9e93dae739448d81c79165f0d0a1c3 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ab9e93dae739448d81c79165f0d0a1c3 2023-07-12 19:12:08,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ab9e93dae739448d81c79165f0d0a1c3, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 19:12:08,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/.tmp/cf2/8014c3f80ce94dccafdd8d11634fbcb3 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/8014c3f80ce94dccafdd8d11634fbcb3 2023-07-12 19:12:08,819 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/8014c3f80ce94dccafdd8d11634fbcb3, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 19:12:08,819 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for f9f0786a2f56ee8f585bc0442b614e60 in 28ms, sequenceid=82, compaction requested=true 2023-07-12 19:12:08,823 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-12 19:12:08,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:12:08,824 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. 2023-07-12 19:12:08,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for f9f0786a2f56ee8f585bc0442b614e60: 2023-07-12 19:12:08,825 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,825 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=f9f0786a2f56ee8f585bc0442b614e60, regionState=CLOSED 2023-07-12 19:12:08,825 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689189128825"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189128825"}]},"ts":"1689189128825"} 2023-07-12 19:12:08,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-07-12 19:12:08,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure f9f0786a2f56ee8f585bc0442b614e60, server=jenkins-hbase20.apache.org,38353,1689189020667 in 187 msec 2023-07-12 19:12:08,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-07-12 19:12:08,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=f9f0786a2f56ee8f585bc0442b614e60, UNASSIGN in 190 msec 2023-07-12 19:12:08,837 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=f9f0786a2f56ee8f585bc0442b614e60, threads=14 2023-07-12 19:12:08,837 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/110063a072624e989128d70308f4d383 for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,837 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/1b04c15cecd5423a9e295067eff9a5ec for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,837 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/3c36f779a1ba409f95aace46e1091f93 for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,837 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/43a2d20956464bad87f3936525e2e4e0 for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,837 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/8014c3f80ce94dccafdd8d11634fbcb3 for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,837 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/a18008abe86b4fabb316a7f979e75f2c for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,837 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/f1ccf904c39e41fcbfa8ed444294f08d for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,838 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/0be45eb572614608a7356cb313a7295f for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,838 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/46a6c3391d354a599ca7cd58973a4fda for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,838 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0 for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,838 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/9906a68f49fe4e3da5382d828bb03e20 for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,838 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ab9e93dae739448d81c79165f0d0a1c3 for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,838 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/e36367a5685b4629984cb2521a64739e for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,838 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ea2e70b6c6934af9b3859991c789e08d for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,979 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/a18008abe86b4fabb316a7f979e75f2c for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,983 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/f1ccf904c39e41fcbfa8ed444294f08d for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:08,998 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ea2e70b6c6934af9b3859991c789e08d for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:09,003 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/43a2d20956464bad87f3936525e2e4e0 for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:09,003 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/110063a072624e989128d70308f4d383 for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:09,006 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/0be45eb572614608a7356cb313a7295f for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:09,010 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/9906a68f49fe4e3da5382d828bb03e20 for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:09,017 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/46a6c3391d354a599ca7cd58973a4fda for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:09,019 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0 for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:09,019 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/3c36f779a1ba409f95aace46e1091f93 for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:09,019 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ab9e93dae739448d81c79165f0d0a1c3 for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:09,025 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/8014c3f80ce94dccafdd8d11634fbcb3 for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:09,030 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/e36367a5685b4629984cb2521a64739e for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:09,373 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/1b04c15cecd5423a9e295067eff9a5ec for region: f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:09,374 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region f9f0786a2f56ee8f585bc0442b614e60 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 19:12:09,402 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 19:12:09,404 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 19:12:09,405 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689189129405"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1689189129405"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1689189129405"}]},"ts":"1689189129405"} 2023-07-12 19:12:09,405 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189129405"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189129405"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189129405"}]},"ts":"1689189129405"} 2023-07-12 19:12:09,405 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189129405"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189129405"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189129405"}]},"ts":"1689189129405"} 2023-07-12 19:12:09,407 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=73093ec69a7f12af7e4d92143adcac5d, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=45dc91a1a73134f94da32c16b90daa3e, ASSIGN}] 2023-07-12 19:12:09,408 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=45dc91a1a73134f94da32c16b90daa3e, ASSIGN 2023-07-12 19:12:09,408 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=45dc91a1a73134f94da32c16b90daa3e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,38353,1689189020667; forceNewPlan=false, retain=false 2023-07-12 19:12:09,408 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=73093ec69a7f12af7e4d92143adcac5d, ASSIGN 2023-07-12 19:12:09,409 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=73093ec69a7f12af7e4d92143adcac5d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,38353,1689189020667; forceNewPlan=false, retain=false 2023-07-12 19:12:09,558 INFO [jenkins-hbase20:43565] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-12 19:12:09,560 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=73093ec69a7f12af7e4d92143adcac5d, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:09,560 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189129559"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189129559"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189129559"}]},"ts":"1689189129559"} 2023-07-12 19:12:09,561 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure 73093ec69a7f12af7e4d92143adcac5d, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:12:09,563 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=45dc91a1a73134f94da32c16b90daa3e, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:09,563 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189129563"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189129563"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189129563"}]},"ts":"1689189129563"} 2023-07-12 19:12:09,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 45dc91a1a73134f94da32c16b90daa3e, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:12:09,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d. 2023-07-12 19:12:09,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 73093ec69a7f12af7e4d92143adcac5d, NAME => 'testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.', STARTKEY => '', ENDKEY => '41'} 2023-07-12 19:12:09,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:12:09,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 73093ec69a7f12af7e4d92143adcac5d 2023-07-12 19:12:09,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:12:09,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 73093ec69a7f12af7e4d92143adcac5d 2023-07-12 19:12:09,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 73093ec69a7f12af7e4d92143adcac5d 2023-07-12 19:12:09,717 INFO [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 73093ec69a7f12af7e4d92143adcac5d 2023-07-12 19:12:09,718 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1 2023-07-12 19:12:09,718 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1 2023-07-12 19:12:09,719 INFO [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 73093ec69a7f12af7e4d92143adcac5d columnFamilyName cf1 2023-07-12 19:12:09,730 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/0be45eb572614608a7356cb313a7295f.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/0be45eb572614608a7356cb313a7295f-bottom 2023-07-12 19:12:09,733 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/46a6c3391d354a599ca7cd58973a4fda.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/46a6c3391d354a599ca7cd58973a4fda-bottom 2023-07-12 19:12:09,737 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0-bottom 2023-07-12 19:12:09,741 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/9906a68f49fe4e3da5382d828bb03e20.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/9906a68f49fe4e3da5382d828bb03e20-bottom 2023-07-12 19:12:09,744 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/ab9e93dae739448d81c79165f0d0a1c3.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ab9e93dae739448d81c79165f0d0a1c3-bottom 2023-07-12 19:12:09,748 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/e36367a5685b4629984cb2521a64739e.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/e36367a5685b4629984cb2521a64739e-bottom 2023-07-12 19:12:09,751 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/ea2e70b6c6934af9b3859991c789e08d.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ea2e70b6c6934af9b3859991c789e08d-bottom 2023-07-12 19:12:09,751 INFO [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(310): Store=73093ec69a7f12af7e4d92143adcac5d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:09,751 INFO [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 73093ec69a7f12af7e4d92143adcac5d 2023-07-12 19:12:09,752 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2 2023-07-12 19:12:09,752 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2 2023-07-12 19:12:09,753 INFO [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 73093ec69a7f12af7e4d92143adcac5d columnFamilyName cf2 2023-07-12 19:12:09,763 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/110063a072624e989128d70308f4d383.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/110063a072624e989128d70308f4d383-bottom 2023-07-12 19:12:09,766 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/1b04c15cecd5423a9e295067eff9a5ec.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/1b04c15cecd5423a9e295067eff9a5ec-bottom 2023-07-12 19:12:09,770 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/3c36f779a1ba409f95aace46e1091f93.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/3c36f779a1ba409f95aace46e1091f93-bottom 2023-07-12 19:12:09,773 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/43a2d20956464bad87f3936525e2e4e0.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/43a2d20956464bad87f3936525e2e4e0-bottom 2023-07-12 19:12:09,776 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/8014c3f80ce94dccafdd8d11634fbcb3.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/8014c3f80ce94dccafdd8d11634fbcb3-bottom 2023-07-12 19:12:09,779 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/a18008abe86b4fabb316a7f979e75f2c.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/a18008abe86b4fabb316a7f979e75f2c-bottom 2023-07-12 19:12:09,782 DEBUG [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/f1ccf904c39e41fcbfa8ed444294f08d.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/f1ccf904c39e41fcbfa8ed444294f08d-bottom 2023-07-12 19:12:09,782 INFO [StoreOpener-73093ec69a7f12af7e4d92143adcac5d-1] regionserver.HStore(310): Store=73093ec69a7f12af7e4d92143adcac5d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:09,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d 2023-07-12 19:12:09,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d 2023-07-12 19:12:09,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:12:09,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 73093ec69a7f12af7e4d92143adcac5d 2023-07-12 19:12:09,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 73093ec69a7f12af7e4d92143adcac5d; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66935046, jitterRate=-0.002590090036392212}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:12:09,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 73093ec69a7f12af7e4d92143adcac5d: 2023-07-12 19:12:09,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d., pid=149, masterSystemTime=1689189129712 2023-07-12 19:12:09,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:12:09,787 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:12:09,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:12:09,787 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:12:09,787 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d. 2023-07-12 19:12:09,787 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d. 2023-07-12 19:12:09,788 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1912): 73093ec69a7f12af7e4d92143adcac5d/cf1 is initiating minor compaction (all files) 2023-07-12 19:12:09,788 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1912): 73093ec69a7f12af7e4d92143adcac5d/cf2 is initiating minor compaction (all files) 2023-07-12 19:12:09,788 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 73093ec69a7f12af7e4d92143adcac5d/cf1 in testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d. 2023-07-12 19:12:09,788 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 73093ec69a7f12af7e4d92143adcac5d/cf2 in testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d. 2023-07-12 19:12:09,788 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/46a6c3391d354a599ca7cd58973a4fda.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/46a6c3391d354a599ca7cd58973a4fda-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/0be45eb572614608a7356cb313a7295f.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/0be45eb572614608a7356cb313a7295f-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/9906a68f49fe4e3da5382d828bb03e20.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/9906a68f49fe4e3da5382d828bb03e20-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/e36367a5685b4629984cb2521a64739e.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/e36367a5685b4629984cb2521a64739e-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/ea2e70b6c6934af9b3859991c789e08d.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ea2e70b6c6934af9b3859991c789e08d-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/ab9e93dae739448d81c79165f0d0a1c3.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ab9e93dae739448d81c79165f0d0a1c3-bottom] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/.tmp, totalSize=34.9 K 2023-07-12 19:12:09,788 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/3c36f779a1ba409f95aace46e1091f93.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/3c36f779a1ba409f95aace46e1091f93-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/110063a072624e989128d70308f4d383.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/110063a072624e989128d70308f4d383-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/f1ccf904c39e41fcbfa8ed444294f08d.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/f1ccf904c39e41fcbfa8ed444294f08d-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/1b04c15cecd5423a9e295067eff9a5ec.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/1b04c15cecd5423a9e295067eff9a5ec-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/a18008abe86b4fabb316a7f979e75f2c.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/a18008abe86b4fabb316a7f979e75f2c-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/43a2d20956464bad87f3936525e2e4e0.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/43a2d20956464bad87f3936525e2e4e0-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/8014c3f80ce94dccafdd8d11634fbcb3.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/8014c3f80ce94dccafdd8d11634fbcb3-bottom] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/.tmp, totalSize=34.9 K 2023-07-12 19:12:09,788 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 46a6c3391d354a599ca7cd58973a4fda.f9f0786a2f56ee8f585bc0442b614e60, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689189127430 2023-07-12 19:12:09,788 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 3c36f779a1ba409f95aace46e1091f93.f9f0786a2f56ee8f585bc0442b614e60, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689189127430 2023-07-12 19:12:09,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d. 2023-07-12 19:12:09,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d. 2023-07-12 19:12:09,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e. 2023-07-12 19:12:09,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 45dc91a1a73134f94da32c16b90daa3e, NAME => 'testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.', STARTKEY => '41', ENDKEY => ''} 2023-07-12 19:12:09,789 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 0be45eb572614608a7356cb313a7295f.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689189127610 2023-07-12 19:12:09,789 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 110063a072624e989128d70308f4d383.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689189127610 2023-07-12 19:12:09,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 19:12:09,789 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 9906a68f49fe4e3da5382d828bb03e20.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689189127783 2023-07-12 19:12:09,789 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting f1ccf904c39e41fcbfa8ed444294f08d.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689189127783 2023-07-12 19:12:09,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:09,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 19:12:09,789 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting e36367a5685b4629984cb2521a64739e.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689189127969 2023-07-12 19:12:09,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:09,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:09,790 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 1b04c15cecd5423a9e295067eff9a5ec.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689189127969 2023-07-12 19:12:09,790 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 6cbbdf997f314ac3bb592f4a9ad1a5d0.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689189128274 2023-07-12 19:12:09,790 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=73093ec69a7f12af7e4d92143adcac5d, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:09,790 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189129790"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189129790"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189129790"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189129790"}]},"ts":"1689189129790"} 2023-07-12 19:12:09,790 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting ea2e70b6c6934af9b3859991c789e08d.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689189128467 2023-07-12 19:12:09,790 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting a18008abe86b4fabb316a7f979e75f2c.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689189128274 2023-07-12 19:12:09,790 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting ab9e93dae739448d81c79165f0d0a1c3.f9f0786a2f56ee8f585bc0442b614e60, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689189128628 2023-07-12 19:12:09,791 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 43a2d20956464bad87f3936525e2e4e0.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689189128467 2023-07-12 19:12:09,791 INFO [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:09,791 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 8014c3f80ce94dccafdd8d11634fbcb3.f9f0786a2f56ee8f585bc0442b614e60, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689189128628 2023-07-12 19:12:09,793 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1 2023-07-12 19:12:09,793 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1 2023-07-12 19:12:09,793 INFO [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 45dc91a1a73134f94da32c16b90daa3e columnFamilyName cf1 2023-07-12 19:12:09,793 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-07-12 19:12:09,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure 73093ec69a7f12af7e4d92143adcac5d, server=jenkins-hbase20.apache.org,38353,1689189020667 in 230 msec 2023-07-12 19:12:09,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=73093ec69a7f12af7e4d92143adcac5d, ASSIGN in 387 msec 2023-07-12 19:12:09,801 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] throttle.PressureAwareThroughputController(145): 73093ec69a7f12af7e4d92143adcac5d#cf1#compaction#140 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:12:09,801 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] throttle.PressureAwareThroughputController(145): 73093ec69a7f12af7e4d92143adcac5d#cf2#compaction#141 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:12:09,804 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/0be45eb572614608a7356cb313a7295f.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/0be45eb572614608a7356cb313a7295f-top 2023-07-12 19:12:09,808 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/46a6c3391d354a599ca7cd58973a4fda.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/46a6c3391d354a599ca7cd58973a4fda-top 2023-07-12 19:12:09,811 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0-top 2023-07-12 19:12:09,814 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/.tmp/cf1/89bb743801154bfc9422e5a8ec0e00bb as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/89bb743801154bfc9422e5a8ec0e00bb 2023-07-12 19:12:09,816 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/9906a68f49fe4e3da5382d828bb03e20.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/9906a68f49fe4e3da5382d828bb03e20-top 2023-07-12 19:12:09,817 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/.tmp/cf2/8d41442a2a7e492c8d2a417a3892a7bc as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/8d41442a2a7e492c8d2a417a3892a7bc 2023-07-12 19:12:09,819 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/ab9e93dae739448d81c79165f0d0a1c3.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ab9e93dae739448d81c79165f0d0a1c3-top 2023-07-12 19:12:09,821 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 73093ec69a7f12af7e4d92143adcac5d/cf1 of 73093ec69a7f12af7e4d92143adcac5d into 89bb743801154bfc9422e5a8ec0e00bb(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:12:09,821 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 73093ec69a7f12af7e4d92143adcac5d/cf2 of 73093ec69a7f12af7e4d92143adcac5d into 8d41442a2a7e492c8d2a417a3892a7bc(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:12:09,821 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 73093ec69a7f12af7e4d92143adcac5d: 2023-07-12 19:12:09,821 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 73093ec69a7f12af7e4d92143adcac5d: 2023-07-12 19:12:09,821 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d., storeName=73093ec69a7f12af7e4d92143adcac5d/cf1, priority=9, startTime=1689189129786; duration=0sec 2023-07-12 19:12:09,821 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d., storeName=73093ec69a7f12af7e4d92143adcac5d/cf2, priority=9, startTime=1689189129787; duration=0sec 2023-07-12 19:12:09,821 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:12:09,821 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:12:09,822 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/e36367a5685b4629984cb2521a64739e.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/e36367a5685b4629984cb2521a64739e-top 2023-07-12 19:12:09,825 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/ea2e70b6c6934af9b3859991c789e08d.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ea2e70b6c6934af9b3859991c789e08d-top 2023-07-12 19:12:09,825 INFO [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(310): Store=45dc91a1a73134f94da32c16b90daa3e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:09,825 INFO [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:09,826 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2 2023-07-12 19:12:09,826 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2 2023-07-12 19:12:09,826 INFO [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 45dc91a1a73134f94da32c16b90daa3e columnFamilyName cf2 2023-07-12 19:12:09,836 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/110063a072624e989128d70308f4d383.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/110063a072624e989128d70308f4d383-top 2023-07-12 19:12:09,838 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/1b04c15cecd5423a9e295067eff9a5ec.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/1b04c15cecd5423a9e295067eff9a5ec-top 2023-07-12 19:12:09,841 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/3c36f779a1ba409f95aace46e1091f93.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/3c36f779a1ba409f95aace46e1091f93-top 2023-07-12 19:12:09,843 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/43a2d20956464bad87f3936525e2e4e0.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/43a2d20956464bad87f3936525e2e4e0-top 2023-07-12 19:12:09,846 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/8014c3f80ce94dccafdd8d11634fbcb3.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/8014c3f80ce94dccafdd8d11634fbcb3-top 2023-07-12 19:12:09,849 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/a18008abe86b4fabb316a7f979e75f2c.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/a18008abe86b4fabb316a7f979e75f2c-top 2023-07-12 19:12:09,852 DEBUG [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/f1ccf904c39e41fcbfa8ed444294f08d.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/f1ccf904c39e41fcbfa8ed444294f08d-top 2023-07-12 19:12:09,852 INFO [StoreOpener-45dc91a1a73134f94da32c16b90daa3e-1] regionserver.HStore(310): Store=45dc91a1a73134f94da32c16b90daa3e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 19:12:09,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:09,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:09,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 19:12:09,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:09,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 45dc91a1a73134f94da32c16b90daa3e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64818305, jitterRate=-0.034131988883018494}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 19:12:09,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 45dc91a1a73134f94da32c16b90daa3e: 2023-07-12 19:12:09,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e., pid=150, masterSystemTime=1689189129712 2023-07-12 19:12:09,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:12:09,857 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 19:12:09,857 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:12:09,857 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 19:12:09,857 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e. 2023-07-12 19:12:09,857 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1912): 45dc91a1a73134f94da32c16b90daa3e/cf1 is initiating minor compaction (all files) 2023-07-12 19:12:09,857 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e. 2023-07-12 19:12:09,857 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1912): 45dc91a1a73134f94da32c16b90daa3e/cf2 is initiating minor compaction (all files) 2023-07-12 19:12:09,857 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 45dc91a1a73134f94da32c16b90daa3e/cf1 in testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e. 2023-07-12 19:12:09,858 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 45dc91a1a73134f94da32c16b90daa3e/cf2 in testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e. 2023-07-12 19:12:09,858 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/46a6c3391d354a599ca7cd58973a4fda.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/46a6c3391d354a599ca7cd58973a4fda-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/0be45eb572614608a7356cb313a7295f.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/0be45eb572614608a7356cb313a7295f-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/9906a68f49fe4e3da5382d828bb03e20.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/9906a68f49fe4e3da5382d828bb03e20-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/e36367a5685b4629984cb2521a64739e.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/e36367a5685b4629984cb2521a64739e-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/ea2e70b6c6934af9b3859991c789e08d.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ea2e70b6c6934af9b3859991c789e08d-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/ab9e93dae739448d81c79165f0d0a1c3.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ab9e93dae739448d81c79165f0d0a1c3-top] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/.tmp, totalSize=34.9 K 2023-07-12 19:12:09,858 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/3c36f779a1ba409f95aace46e1091f93.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/3c36f779a1ba409f95aace46e1091f93-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/110063a072624e989128d70308f4d383.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/110063a072624e989128d70308f4d383-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/f1ccf904c39e41fcbfa8ed444294f08d.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/f1ccf904c39e41fcbfa8ed444294f08d-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/1b04c15cecd5423a9e295067eff9a5ec.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/1b04c15cecd5423a9e295067eff9a5ec-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/a18008abe86b4fabb316a7f979e75f2c.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/a18008abe86b4fabb316a7f979e75f2c-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/43a2d20956464bad87f3936525e2e4e0.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/43a2d20956464bad87f3936525e2e4e0-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/8014c3f80ce94dccafdd8d11634fbcb3.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/8014c3f80ce94dccafdd8d11634fbcb3-top] into tmpdir=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/.tmp, totalSize=34.9 K 2023-07-12 19:12:09,858 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 46a6c3391d354a599ca7cd58973a4fda.f9f0786a2f56ee8f585bc0442b614e60, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689189127430 2023-07-12 19:12:09,858 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 3c36f779a1ba409f95aace46e1091f93.f9f0786a2f56ee8f585bc0442b614e60, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689189127430 2023-07-12 19:12:09,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e. 2023-07-12 19:12:09,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e. 2023-07-12 19:12:09,859 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 0be45eb572614608a7356cb313a7295f.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689189127610 2023-07-12 19:12:09,859 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 110063a072624e989128d70308f4d383.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689189127610 2023-07-12 19:12:09,859 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=45dc91a1a73134f94da32c16b90daa3e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:09,859 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189129859"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689189129859"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689189129859"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689189129859"}]},"ts":"1689189129859"} 2023-07-12 19:12:09,859 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 9906a68f49fe4e3da5382d828bb03e20.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689189127783 2023-07-12 19:12:09,859 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting f1ccf904c39e41fcbfa8ed444294f08d.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689189127783 2023-07-12 19:12:09,859 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting e36367a5685b4629984cb2521a64739e.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689189127969 2023-07-12 19:12:09,859 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 1b04c15cecd5423a9e295067eff9a5ec.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689189127969 2023-07-12 19:12:09,860 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting 6cbbdf997f314ac3bb592f4a9ad1a5d0.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1689189128274 2023-07-12 19:12:09,860 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting a18008abe86b4fabb316a7f979e75f2c.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1689189128274 2023-07-12 19:12:09,860 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting ea2e70b6c6934af9b3859991c789e08d.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1689189128467 2023-07-12 19:12:09,860 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 43a2d20956464bad87f3936525e2e4e0.f9f0786a2f56ee8f585bc0442b614e60, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1689189128467 2023-07-12 19:12:09,860 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] compactions.Compactor(207): Compacting ab9e93dae739448d81c79165f0d0a1c3.f9f0786a2f56ee8f585bc0442b614e60, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689189128628 2023-07-12 19:12:09,860 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] compactions.Compactor(207): Compacting 8014c3f80ce94dccafdd8d11634fbcb3.f9f0786a2f56ee8f585bc0442b614e60, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689189128628 2023-07-12 19:12:09,861 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-07-12 19:12:09,861 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 45dc91a1a73134f94da32c16b90daa3e, server=jenkins-hbase20.apache.org,38353,1689189020667 in 297 msec 2023-07-12 19:12:09,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-07-12 19:12:09,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=45dc91a1a73134f94da32c16b90daa3e, ASSIGN in 454 msec 2023-07-12 19:12:09,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=f9f0786a2f56ee8f585bc0442b614e60, daughterA=73093ec69a7f12af7e4d92143adcac5d, daughterB=45dc91a1a73134f94da32c16b90daa3e in 1.2280 sec 2023-07-12 19:12:09,868 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] throttle.PressureAwareThroughputController(145): 45dc91a1a73134f94da32c16b90daa3e#cf1#compaction#142 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:12:09,868 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] throttle.PressureAwareThroughputController(145): 45dc91a1a73134f94da32c16b90daa3e#cf2#compaction#143 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 19:12:09,882 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/.tmp/cf2/d2a24446e15e4916b55e913df909f8f9 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/d2a24446e15e4916b55e913df909f8f9 2023-07-12 19:12:09,883 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/.tmp/cf1/bd3bda0d50b64a118879ac536f285c5a as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/bd3bda0d50b64a118879ac536f285c5a 2023-07-12 19:12:09,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38353] ipc.CallRunner(144): callId: 1118 service: ClientService methodName: Scan size: 143 connection: 148.251.75.209:59964 deadline: 1689189189886, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60. is not online on jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:09,886 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 45dc91a1a73134f94da32c16b90daa3e/cf2 of 45dc91a1a73134f94da32c16b90daa3e into d2a24446e15e4916b55e913df909f8f9(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:12:09,886 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 45dc91a1a73134f94da32c16b90daa3e: 2023-07-12 19:12:09,886 INFO [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e., storeName=45dc91a1a73134f94da32c16b90daa3e/cf2, priority=9, startTime=1689189129857; duration=0sec 2023-07-12 19:12:09,887 DEBUG [RS:2;jenkins-hbase20:38353-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:12:09,888 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 45dc91a1a73134f94da32c16b90daa3e/cf1 of 45dc91a1a73134f94da32c16b90daa3e into bd3bda0d50b64a118879ac536f285c5a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 19:12:09,888 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 45dc91a1a73134f94da32c16b90daa3e: 2023-07-12 19:12:09,888 INFO [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e., storeName=45dc91a1a73134f94da32c16b90daa3e/cf1, priority=9, startTime=1689189129856; duration=0sec 2023-07-12 19:12:09,889 DEBUG [RS:2;jenkins-hbase20:38353-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 19:12:09,995 DEBUG [Listener at localhost.localdomain/40405] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-12 19:12:10,001 INFO [Listener at localhost.localdomain/40405] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-12 19:12:10,003 DEBUG [Listener at localhost.localdomain/40405] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-12 19:12:10,009 WARN [Listener at localhost.localdomain/40405] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 19:12:10,010 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-07-12 19:12:10,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitTableRegion 2023-07-12 19:12:10,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-07-12 19:12:10,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-07-12 19:12:10,014 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189130014"}]},"ts":"1689189130014"} 2023-07-12 19:12:10,014 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-07-12 19:12:10,015 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-07-12 19:12:10,016 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=73093ec69a7f12af7e4d92143adcac5d, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=45dc91a1a73134f94da32c16b90daa3e, UNASSIGN}] 2023-07-12 19:12:10,016 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=45dc91a1a73134f94da32c16b90daa3e, UNASSIGN 2023-07-12 19:12:10,017 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=45dc91a1a73134f94da32c16b90daa3e, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:10,017 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189130017"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189130017"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189130017"}]},"ts":"1689189130017"} 2023-07-12 19:12:10,018 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 45dc91a1a73134f94da32c16b90daa3e, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:12:10,019 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=73093ec69a7f12af7e4d92143adcac5d, UNASSIGN 2023-07-12 19:12:10,020 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=73093ec69a7f12af7e4d92143adcac5d, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:10,020 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189130020"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689189130020"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689189130020"}]},"ts":"1689189130020"} 2023-07-12 19:12:10,022 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 73093ec69a7f12af7e4d92143adcac5d, server=jenkins-hbase20.apache.org,38353,1689189020667}] 2023-07-12 19:12:10,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-07-12 19:12:10,169 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 73093ec69a7f12af7e4d92143adcac5d 2023-07-12 19:12:10,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 73093ec69a7f12af7e4d92143adcac5d, disabling compactions & flushes 2023-07-12 19:12:10,169 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d. 2023-07-12 19:12:10,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d. 2023-07-12 19:12:10,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d. after waiting 0 ms 2023-07-12 19:12:10,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d. 2023-07-12 19:12:10,170 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/46a6c3391d354a599ca7cd58973a4fda.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/46a6c3391d354a599ca7cd58973a4fda-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/0be45eb572614608a7356cb313a7295f.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/0be45eb572614608a7356cb313a7295f-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/9906a68f49fe4e3da5382d828bb03e20.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/9906a68f49fe4e3da5382d828bb03e20-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/e36367a5685b4629984cb2521a64739e.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/e36367a5685b4629984cb2521a64739e-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/ea2e70b6c6934af9b3859991c789e08d.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ea2e70b6c6934af9b3859991c789e08d-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/ab9e93dae739448d81c79165f0d0a1c3.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ab9e93dae739448d81c79165f0d0a1c3-bottom] to archive 2023-07-12 19:12:10,170 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:12:10,172 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/46a6c3391d354a599ca7cd58973a4fda.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/46a6c3391d354a599ca7cd58973a4fda.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,173 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/0be45eb572614608a7356cb313a7295f.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/0be45eb572614608a7356cb313a7295f.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,174 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/9906a68f49fe4e3da5382d828bb03e20.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/9906a68f49fe4e3da5382d828bb03e20.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,175 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/e36367a5685b4629984cb2521a64739e.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/e36367a5685b4629984cb2521a64739e.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,176 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,177 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/ea2e70b6c6934af9b3859991c789e08d.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/ea2e70b6c6934af9b3859991c789e08d.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,178 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/ab9e93dae739448d81c79165f0d0a1c3.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/ab9e93dae739448d81c79165f0d0a1c3.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,179 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/3c36f779a1ba409f95aace46e1091f93.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/3c36f779a1ba409f95aace46e1091f93-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/110063a072624e989128d70308f4d383.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/110063a072624e989128d70308f4d383-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/f1ccf904c39e41fcbfa8ed444294f08d.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/f1ccf904c39e41fcbfa8ed444294f08d-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/1b04c15cecd5423a9e295067eff9a5ec.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/1b04c15cecd5423a9e295067eff9a5ec-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/a18008abe86b4fabb316a7f979e75f2c.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/a18008abe86b4fabb316a7f979e75f2c-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/43a2d20956464bad87f3936525e2e4e0.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/43a2d20956464bad87f3936525e2e4e0-bottom, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/8014c3f80ce94dccafdd8d11634fbcb3.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/8014c3f80ce94dccafdd8d11634fbcb3-bottom] to archive 2023-07-12 19:12:10,180 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:12:10,181 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/3c36f779a1ba409f95aace46e1091f93.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/3c36f779a1ba409f95aace46e1091f93.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,181 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/110063a072624e989128d70308f4d383.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/110063a072624e989128d70308f4d383.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,182 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/f1ccf904c39e41fcbfa8ed444294f08d.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/f1ccf904c39e41fcbfa8ed444294f08d.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,183 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/1b04c15cecd5423a9e295067eff9a5ec.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/1b04c15cecd5423a9e295067eff9a5ec.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,184 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/a18008abe86b4fabb316a7f979e75f2c.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/a18008abe86b4fabb316a7f979e75f2c.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,185 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/43a2d20956464bad87f3936525e2e4e0.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/43a2d20956464bad87f3936525e2e4e0.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,185 DEBUG [StoreCloser-testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/8014c3f80ce94dccafdd8d11634fbcb3.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/8014c3f80ce94dccafdd8d11634fbcb3.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 19:12:10,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:12:10,189 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d. 2023-07-12 19:12:10,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 73093ec69a7f12af7e4d92143adcac5d: 2023-07-12 19:12:10,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 73093ec69a7f12af7e4d92143adcac5d 2023-07-12 19:12:10,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:10,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 45dc91a1a73134f94da32c16b90daa3e, disabling compactions & flushes 2023-07-12 19:12:10,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e. 2023-07-12 19:12:10,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e. 2023-07-12 19:12:10,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e. after waiting 0 ms 2023-07-12 19:12:10,190 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=73093ec69a7f12af7e4d92143adcac5d, regionState=CLOSED 2023-07-12 19:12:10,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e. 2023-07-12 19:12:10,190 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189130190"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189130190"}]},"ts":"1689189130190"} 2023-07-12 19:12:10,191 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/46a6c3391d354a599ca7cd58973a4fda.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/46a6c3391d354a599ca7cd58973a4fda-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/0be45eb572614608a7356cb313a7295f.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/0be45eb572614608a7356cb313a7295f-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/9906a68f49fe4e3da5382d828bb03e20.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/9906a68f49fe4e3da5382d828bb03e20-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/e36367a5685b4629984cb2521a64739e.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/e36367a5685b4629984cb2521a64739e-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/ea2e70b6c6934af9b3859991c789e08d.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ea2e70b6c6934af9b3859991c789e08d-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/ab9e93dae739448d81c79165f0d0a1c3.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ab9e93dae739448d81c79165f0d0a1c3-top] to archive 2023-07-12 19:12:10,191 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:12:10,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-07-12 19:12:10,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 73093ec69a7f12af7e4d92143adcac5d, server=jenkins-hbase20.apache.org,38353,1689189020667 in 170 msec 2023-07-12 19:12:10,193 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/46a6c3391d354a599ca7cd58973a4fda.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/46a6c3391d354a599ca7cd58973a4fda.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=73093ec69a7f12af7e4d92143adcac5d, UNASSIGN in 176 msec 2023-07-12 19:12:10,193 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/0be45eb572614608a7356cb313a7295f.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/0be45eb572614608a7356cb313a7295f.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,194 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/9906a68f49fe4e3da5382d828bb03e20.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/9906a68f49fe4e3da5382d828bb03e20.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,195 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/e36367a5685b4629984cb2521a64739e.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/e36367a5685b4629984cb2521a64739e.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,196 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,197 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/ea2e70b6c6934af9b3859991c789e08d.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/ea2e70b6c6934af9b3859991c789e08d.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,197 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/ab9e93dae739448d81c79165f0d0a1c3.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/ab9e93dae739448d81c79165f0d0a1c3.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,199 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/3c36f779a1ba409f95aace46e1091f93.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/3c36f779a1ba409f95aace46e1091f93-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/110063a072624e989128d70308f4d383.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/110063a072624e989128d70308f4d383-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/f1ccf904c39e41fcbfa8ed444294f08d.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/f1ccf904c39e41fcbfa8ed444294f08d-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/1b04c15cecd5423a9e295067eff9a5ec.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/1b04c15cecd5423a9e295067eff9a5ec-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/a18008abe86b4fabb316a7f979e75f2c.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/a18008abe86b4fabb316a7f979e75f2c-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/43a2d20956464bad87f3936525e2e4e0.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/43a2d20956464bad87f3936525e2e4e0-top, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/8014c3f80ce94dccafdd8d11634fbcb3.f9f0786a2f56ee8f585bc0442b614e60->hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/8014c3f80ce94dccafdd8d11634fbcb3-top] to archive 2023-07-12 19:12:10,199 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 19:12:10,200 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/3c36f779a1ba409f95aace46e1091f93.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/3c36f779a1ba409f95aace46e1091f93.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,201 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/110063a072624e989128d70308f4d383.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/110063a072624e989128d70308f4d383.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,202 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/f1ccf904c39e41fcbfa8ed444294f08d.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/f1ccf904c39e41fcbfa8ed444294f08d.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,202 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/1b04c15cecd5423a9e295067eff9a5ec.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/1b04c15cecd5423a9e295067eff9a5ec.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,203 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/a18008abe86b4fabb316a7f979e75f2c.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/a18008abe86b4fabb316a7f979e75f2c.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,204 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/43a2d20956464bad87f3936525e2e4e0.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/43a2d20956464bad87f3936525e2e4e0.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,204 DEBUG [StoreCloser-testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/8014c3f80ce94dccafdd8d11634fbcb3.f9f0786a2f56ee8f585bc0442b614e60 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/8014c3f80ce94dccafdd8d11634fbcb3.f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 19:12:10,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:12:10,208 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e. 2023-07-12 19:12:10,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 45dc91a1a73134f94da32c16b90daa3e: 2023-07-12 19:12:10,209 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:10,209 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=45dc91a1a73134f94da32c16b90daa3e, regionState=CLOSED 2023-07-12 19:12:10,209 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689189130209"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689189130209"}]},"ts":"1689189130209"} 2023-07-12 19:12:10,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-07-12 19:12:10,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 45dc91a1a73134f94da32c16b90daa3e, server=jenkins-hbase20.apache.org,38353,1689189020667 in 193 msec 2023-07-12 19:12:10,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-07-12 19:12:10,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=45dc91a1a73134f94da32c16b90daa3e, UNASSIGN in 195 msec 2023-07-12 19:12:10,212 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689189130212"}]},"ts":"1689189130212"} 2023-07-12 19:12:10,213 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-07-12 19:12:10,213 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-07-12 19:12:10,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 203 msec 2023-07-12 19:12:10,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-07-12 19:12:10,315 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-07-12 19:12:10,316 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitTableRegion 2023-07-12 19:12:10,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-07-12 19:12:10,318 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-12 19:12:10,319 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-12 19:12:10,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-07-12 19:12:10,322 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,322 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:10,322 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d 2023-07-12 19:12:10,323 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/recovered.edits] 2023-07-12 19:12:10,323 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/recovered.edits] 2023-07-12 19:12:10,323 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2, FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/recovered.edits] 2023-07-12 19:12:10,325 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/89bb743801154bfc9422e5a8ec0e00bb to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf1/89bb743801154bfc9422e5a8ec0e00bb 2023-07-12 19:12:10,326 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/bd3bda0d50b64a118879ac536f285c5a to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf1/bd3bda0d50b64a118879ac536f285c5a 2023-07-12 19:12:10,326 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/0be45eb572614608a7356cb313a7295f to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/0be45eb572614608a7356cb313a7295f 2023-07-12 19:12:10,327 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/8d41442a2a7e492c8d2a417a3892a7bc to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/cf2/8d41442a2a7e492c8d2a417a3892a7bc 2023-07-12 19:12:10,328 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/46a6c3391d354a599ca7cd58973a4fda to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/46a6c3391d354a599ca7cd58973a4fda 2023-07-12 19:12:10,328 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/d2a24446e15e4916b55e913df909f8f9 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/cf2/d2a24446e15e4916b55e913df909f8f9 2023-07-12 19:12:10,329 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/6cbbdf997f314ac3bb592f4a9ad1a5d0 2023-07-12 19:12:10,329 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/recovered.edits/90.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d/recovered.edits/90.seqid 2023-07-12 19:12:10,329 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/9906a68f49fe4e3da5382d828bb03e20 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/9906a68f49fe4e3da5382d828bb03e20 2023-07-12 19:12:10,330 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/recovered.edits/90.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e/recovered.edits/90.seqid 2023-07-12 19:12:10,330 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/73093ec69a7f12af7e4d92143adcac5d 2023-07-12 19:12:10,330 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/45dc91a1a73134f94da32c16b90daa3e 2023-07-12 19:12:10,331 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ab9e93dae739448d81c79165f0d0a1c3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ab9e93dae739448d81c79165f0d0a1c3 2023-07-12 19:12:10,331 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/e36367a5685b4629984cb2521a64739e to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/e36367a5685b4629984cb2521a64739e 2023-07-12 19:12:10,332 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ea2e70b6c6934af9b3859991c789e08d to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf1/ea2e70b6c6934af9b3859991c789e08d 2023-07-12 19:12:10,334 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/110063a072624e989128d70308f4d383 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/110063a072624e989128d70308f4d383 2023-07-12 19:12:10,335 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/1b04c15cecd5423a9e295067eff9a5ec to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/1b04c15cecd5423a9e295067eff9a5ec 2023-07-12 19:12:10,335 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/3c36f779a1ba409f95aace46e1091f93 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/3c36f779a1ba409f95aace46e1091f93 2023-07-12 19:12:10,336 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/43a2d20956464bad87f3936525e2e4e0 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/43a2d20956464bad87f3936525e2e4e0 2023-07-12 19:12:10,337 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/8014c3f80ce94dccafdd8d11634fbcb3 to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/8014c3f80ce94dccafdd8d11634fbcb3 2023-07-12 19:12:10,337 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/a18008abe86b4fabb316a7f979e75f2c to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/a18008abe86b4fabb316a7f979e75f2c 2023-07-12 19:12:10,338 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/f1ccf904c39e41fcbfa8ed444294f08d to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/cf2/f1ccf904c39e41fcbfa8ed444294f08d 2023-07-12 19:12:10,340 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/recovered.edits/85.seqid to hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/archive/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60/recovered.edits/85.seqid 2023-07-12 19:12:10,340 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/.tmp/data/default/testSplitTableRegion/f9f0786a2f56ee8f585bc0442b614e60 2023-07-12 19:12:10,340 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-07-12 19:12:10,342 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-12 19:12:10,343 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-07-12 19:12:10,344 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-07-12 19:12:10,345 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-12 19:12:10,345 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-07-12 19:12:10,345 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189130345"}]},"ts":"9223372036854775807"} 2023-07-12 19:12:10,345 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189130345"}]},"ts":"9223372036854775807"} 2023-07-12 19:12:10,346 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689189130345"}]},"ts":"9223372036854775807"} 2023-07-12 19:12:10,347 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-12 19:12:10,347 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => f9f0786a2f56ee8f585bc0442b614e60, NAME => 'testSplitTableRegion,,1689189126924.f9f0786a2f56ee8f585bc0442b614e60.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 73093ec69a7f12af7e4d92143adcac5d, NAME => 'testSplitTableRegion,,1689189128634.73093ec69a7f12af7e4d92143adcac5d.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 45dc91a1a73134f94da32c16b90daa3e, NAME => 'testSplitTableRegion,41,1689189128634.45dc91a1a73134f94da32c16b90daa3e.', STARTKEY => '41', ENDKEY => ''}] 2023-07-12 19:12:10,347 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-07-12 19:12:10,347 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689189130347"}]},"ts":"9223372036854775807"} 2023-07-12 19:12:10,348 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-07-12 19:12:10,349 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-12 19:12:10,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 34 msec 2023-07-12 19:12:10,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43565] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-07-12 19:12:10,420 INFO [Listener at localhost.localdomain/40405] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-07-12 19:12:10,431 INFO [Listener at localhost.localdomain/40405] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=378 (was 375) - Thread LEAK? -, OpenFileDescriptor=820 (was 833), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=87 (was 85) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=5641 (was 5727) 2023-07-12 19:12:10,431 INFO [Listener at localhost.localdomain/40405] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-12 19:12:10,435 INFO [Listener at localhost.localdomain/40405] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-12 19:12:10,435 DEBUG [Listener at localhost.localdomain/40405] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x33996176 to 127.0.0.1:63235 2023-07-12 19:12:10,435 DEBUG [Listener at localhost.localdomain/40405] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 19:12:10,436 DEBUG [Listener at localhost.localdomain/40405] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-12 19:12:10,436 DEBUG [Listener at localhost.localdomain/40405] util.JVMClusterUtil(257): Found active master hash=1091277552, stopped=false 2023-07-12 19:12:10,436 INFO [Listener at localhost.localdomain/40405] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase20.apache.org,43565,1689189019651 2023-07-12 19:12:10,437 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-12 19:12:10,437 INFO [Listener at localhost.localdomain/40405] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 19:12:10,437 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 19:12:10,437 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-12 19:12:10,437 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-12 19:12:10,437 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-12 19:12:10,437 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 19:12:10,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 19:12:10,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 19:12:10,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 19:12:10,438 DEBUG [Listener at localhost.localdomain/40405] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x46e118df to 127.0.0.1:63235 2023-07-12 19:12:10,438 DEBUG [Listener at localhost.localdomain/40405] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 19:12:10,438 INFO [Listener at localhost.localdomain/40405] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase20.apache.org,45655,1689189020596' ***** 2023-07-12 19:12:10,438 INFO [Listener at localhost.localdomain/40405] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-12 19:12:10,438 INFO [Listener at localhost.localdomain/40405] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase20.apache.org,33255,1689189020635' ***** 2023-07-12 19:12:10,438 INFO [Listener at localhost.localdomain/40405] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-12 19:12:10,438 INFO [Listener at localhost.localdomain/40405] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase20.apache.org,38353,1689189020667' ***** 2023-07-12 19:12:10,438 INFO [Listener at localhost.localdomain/40405] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-12 19:12:10,438 INFO [RS:1;jenkins-hbase20:33255] regionserver.HeapMemoryManager(220): Stopping 2023-07-12 19:12:10,438 INFO [RS:2;jenkins-hbase20:38353] regionserver.HeapMemoryManager(220): Stopping 2023-07-12 19:12:10,439 INFO [RS:1;jenkins-hbase20:33255] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-12 19:12:10,439 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-12 19:12:10,439 INFO [RS:1;jenkins-hbase20:33255] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-12 19:12:10,439 INFO [RS:2;jenkins-hbase20:38353] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-12 19:12:10,439 INFO [RS:1;jenkins-hbase20:33255] regionserver.HRegionServer(1144): stopping server jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:10,439 INFO [RS:0;jenkins-hbase20:45655] regionserver.HeapMemoryManager(220): Stopping 2023-07-12 19:12:10,439 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-12 19:12:10,439 INFO [RS:0;jenkins-hbase20:45655] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-12 19:12:10,439 INFO [RS:2;jenkins-hbase20:38353] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-12 19:12:10,439 DEBUG [RS:1;jenkins-hbase20:33255] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5b5b3756 to 127.0.0.1:63235 2023-07-12 19:12:10,439 INFO [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer(1144): stopping server jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:10,439 DEBUG [RS:2;jenkins-hbase20:38353] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x03c2fe17 to 127.0.0.1:63235 2023-07-12 19:12:10,439 DEBUG [RS:2;jenkins-hbase20:38353] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 19:12:10,439 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-12 19:12:10,439 INFO [RS:0;jenkins-hbase20:45655] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-12 19:12:10,439 INFO [RS:2;jenkins-hbase20:38353] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-12 19:12:10,440 INFO [RS:2;jenkins-hbase20:38353] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-12 19:12:10,439 DEBUG [RS:1;jenkins-hbase20:33255] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 19:12:10,440 INFO [RS:2;jenkins-hbase20:38353] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-12 19:12:10,440 INFO [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer(3305): Received CLOSE for da4636671d8cf580eb751750528e08a9 2023-07-12 19:12:10,440 INFO [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-12 19:12:10,440 INFO [RS:1;jenkins-hbase20:33255] regionserver.HRegionServer(1170): stopping server jenkins-hbase20.apache.org,33255,1689189020635; all regions closed. 2023-07-12 19:12:10,440 INFO [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer(1144): stopping server jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:10,440 DEBUG [RS:0;jenkins-hbase20:45655] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5a50683e to 127.0.0.1:63235 2023-07-12 19:12:10,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing da4636671d8cf580eb751750528e08a9, disabling compactions & flushes 2023-07-12 19:12:10,440 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9. 2023-07-12 19:12:10,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9. 2023-07-12 19:12:10,440 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-12 19:12:10,440 INFO [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-12 19:12:10,440 DEBUG [RS:0;jenkins-hbase20:45655] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 19:12:10,440 DEBUG [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-07-12 19:12:10,440 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-12 19:12:10,440 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-12 19:12:10,440 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-12 19:12:10,440 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-12 19:12:10,441 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=79.49 KB heapSize=125.82 KB 2023-07-12 19:12:10,441 DEBUG [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-12 19:12:10,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9. after waiting 0 ms 2023-07-12 19:12:10,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9. 2023-07-12 19:12:10,440 INFO [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-12 19:12:10,441 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing da4636671d8cf580eb751750528e08a9 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-12 19:12:10,441 DEBUG [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer(1478): Online Regions={da4636671d8cf580eb751750528e08a9=hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9.} 2023-07-12 19:12:10,441 DEBUG [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer(1504): Waiting on da4636671d8cf580eb751750528e08a9 2023-07-12 19:12:10,453 DEBUG [RS:1;jenkins-hbase20:33255] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/oldWALs 2023-07-12 19:12:10,453 INFO [RS:1;jenkins-hbase20:33255] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C33255%2C1689189020635:(num 1689189022042) 2023-07-12 19:12:10,453 DEBUG [RS:1;jenkins-hbase20:33255] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 19:12:10,453 INFO [RS:1;jenkins-hbase20:33255] regionserver.LeaseManager(133): Closed leases 2023-07-12 19:12:10,458 INFO [RS:1;jenkins-hbase20:33255] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-12 19:12:10,458 INFO [RS:1;jenkins-hbase20:33255] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-12 19:12:10,458 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 19:12:10,458 INFO [RS:1;jenkins-hbase20:33255] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-12 19:12:10,458 INFO [RS:1;jenkins-hbase20:33255] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-12 19:12:10,462 INFO [RS:1;jenkins-hbase20:33255] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:33255 2023-07-12 19:12:10,466 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/namespace/da4636671d8cf580eb751750528e08a9/.tmp/info/f3c41c5dee9a4899a7bc957a01fc98d7 2023-07-12 19:12:10,466 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:10,466 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-12 19:12:10,466 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:10,466 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,33255,1689189020635 2023-07-12 19:12:10,466 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-12 19:12:10,466 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-12 19:12:10,466 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-12 19:12:10,467 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,33255,1689189020635] 2023-07-12 19:12:10,467 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,33255,1689189020635; numProcessing=1 2023-07-12 19:12:10,468 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,33255,1689189020635 already deleted, retry=false 2023-07-12 19:12:10,468 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,33255,1689189020635 expired; onlineServers=2 2023-07-12 19:12:10,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/namespace/da4636671d8cf580eb751750528e08a9/.tmp/info/f3c41c5dee9a4899a7bc957a01fc98d7 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/namespace/da4636671d8cf580eb751750528e08a9/info/f3c41c5dee9a4899a7bc957a01fc98d7 2023-07-12 19:12:10,474 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/namespace/da4636671d8cf580eb751750528e08a9/info/f3c41c5dee9a4899a7bc957a01fc98d7, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 19:12:10,475 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for da4636671d8cf580eb751750528e08a9 in 34ms, sequenceid=6, compaction requested=false 2023-07-12 19:12:10,482 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.37 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/.tmp/info/0207fb3a9b2a4f41880eca98cb86c50a 2023-07-12 19:12:10,482 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/namespace/da4636671d8cf580eb751750528e08a9/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-12 19:12:10,482 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:12:10,483 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9. 2023-07-12 19:12:10,483 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for da4636671d8cf580eb751750528e08a9: 2023-07-12 19:12:10,483 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689189022527.da4636671d8cf580eb751750528e08a9. 2023-07-12 19:12:10,486 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0207fb3a9b2a4f41880eca98cb86c50a 2023-07-12 19:12:10,497 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/.tmp/rep_barrier/b2a7a5ab39314d47819a095a6c975b03 2023-07-12 19:12:10,500 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b2a7a5ab39314d47819a095a6c975b03 2023-07-12 19:12:10,509 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/.tmp/table/414262a0dfae431f8f235bbb652e8450 2023-07-12 19:12:10,512 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 414262a0dfae431f8f235bbb652e8450 2023-07-12 19:12:10,513 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/.tmp/info/0207fb3a9b2a4f41880eca98cb86c50a as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/info/0207fb3a9b2a4f41880eca98cb86c50a 2023-07-12 19:12:10,516 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0207fb3a9b2a4f41880eca98cb86c50a 2023-07-12 19:12:10,516 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/info/0207fb3a9b2a4f41880eca98cb86c50a, entries=59, sequenceid=210, filesize=11.1 K 2023-07-12 19:12:10,517 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/.tmp/rep_barrier/b2a7a5ab39314d47819a095a6c975b03 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/rep_barrier/b2a7a5ab39314d47819a095a6c975b03 2023-07-12 19:12:10,518 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-12 19:12:10,518 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-12 19:12:10,518 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-12 19:12:10,520 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b2a7a5ab39314d47819a095a6c975b03 2023-07-12 19:12:10,520 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/rep_barrier/b2a7a5ab39314d47819a095a6c975b03, entries=24, sequenceid=210, filesize=7.5 K 2023-07-12 19:12:10,521 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/.tmp/table/414262a0dfae431f8f235bbb652e8450 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/table/414262a0dfae431f8f235bbb652e8450 2023-07-12 19:12:10,525 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 414262a0dfae431f8f235bbb652e8450 2023-07-12 19:12:10,525 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/table/414262a0dfae431f8f235bbb652e8450, entries=36, sequenceid=210, filesize=8.0 K 2023-07-12 19:12:10,525 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 85ms, sequenceid=210, compaction requested=false 2023-07-12 19:12:10,529 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-07-12 19:12:10,530 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 19:12:10,530 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-12 19:12:10,531 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-12 19:12:10,531 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-12 19:12:10,531 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-12 19:12:10,567 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 19:12:10,568 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:33255-0x100829769890002, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 19:12:10,567 INFO [RS:1;jenkins-hbase20:33255] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase20.apache.org,33255,1689189020635; zookeeper connection closed. 2023-07-12 19:12:10,568 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1f9951ee] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1f9951ee 2023-07-12 19:12:10,641 INFO [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer(1170): stopping server jenkins-hbase20.apache.org,38353,1689189020667; all regions closed. 2023-07-12 19:12:10,641 INFO [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer(1170): stopping server jenkins-hbase20.apache.org,45655,1689189020596; all regions closed. 2023-07-12 19:12:10,645 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/WALs/jenkins-hbase20.apache.org,38353,1689189020667/jenkins-hbase20.apache.org%2C38353%2C1689189020667.meta.1689189022248.meta not finished, retry = 0 2023-07-12 19:12:10,648 DEBUG [RS:0;jenkins-hbase20:45655] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/oldWALs 2023-07-12 19:12:10,648 INFO [RS:0;jenkins-hbase20:45655] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C45655%2C1689189020596:(num 1689189022042) 2023-07-12 19:12:10,648 DEBUG [RS:0;jenkins-hbase20:45655] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 19:12:10,648 INFO [RS:0;jenkins-hbase20:45655] regionserver.LeaseManager(133): Closed leases 2023-07-12 19:12:10,648 INFO [RS:0;jenkins-hbase20:45655] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-12 19:12:10,649 INFO [RS:0;jenkins-hbase20:45655] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-12 19:12:10,649 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 19:12:10,649 INFO [RS:0;jenkins-hbase20:45655] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-12 19:12:10,649 INFO [RS:0;jenkins-hbase20:45655] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-12 19:12:10,649 INFO [RS:0;jenkins-hbase20:45655] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:45655 2023-07-12 19:12:10,650 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:10,650 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-12 19:12:10,650 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,45655,1689189020596 2023-07-12 19:12:10,651 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,45655,1689189020596] 2023-07-12 19:12:10,651 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,45655,1689189020596; numProcessing=2 2023-07-12 19:12:10,651 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,45655,1689189020596 already deleted, retry=false 2023-07-12 19:12:10,651 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,45655,1689189020596 expired; onlineServers=1 2023-07-12 19:12:10,748 DEBUG [RS:2;jenkins-hbase20:38353] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/oldWALs 2023-07-12 19:12:10,748 INFO [RS:2;jenkins-hbase20:38353] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C38353%2C1689189020667.meta:.meta(num 1689189022248) 2023-07-12 19:12:10,751 INFO [RS:0;jenkins-hbase20:45655] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase20.apache.org,45655,1689189020596; zookeeper connection closed. 2023-07-12 19:12:10,751 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 19:12:10,751 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:45655-0x100829769890001, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 19:12:10,752 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@53aab15f] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@53aab15f 2023-07-12 19:12:10,752 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/WALs/jenkins-hbase20.apache.org,38353,1689189020667/jenkins-hbase20.apache.org%2C38353%2C1689189020667.1689189022043 not finished, retry = 0 2023-07-12 19:12:10,854 DEBUG [RS:2;jenkins-hbase20:38353] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/oldWALs 2023-07-12 19:12:10,854 INFO [RS:2;jenkins-hbase20:38353] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C38353%2C1689189020667:(num 1689189022043) 2023-07-12 19:12:10,854 DEBUG [RS:2;jenkins-hbase20:38353] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 19:12:10,854 INFO [RS:2;jenkins-hbase20:38353] regionserver.LeaseManager(133): Closed leases 2023-07-12 19:12:10,855 INFO [RS:2;jenkins-hbase20:38353] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-12 19:12:10,855 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 19:12:10,856 INFO [RS:2;jenkins-hbase20:38353] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:38353 2023-07-12 19:12:10,857 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,38353,1689189020667 2023-07-12 19:12:10,857 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-12 19:12:10,858 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,38353,1689189020667] 2023-07-12 19:12:10,858 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,38353,1689189020667; numProcessing=3 2023-07-12 19:12:10,858 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,38353,1689189020667 already deleted, retry=false 2023-07-12 19:12:10,858 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,38353,1689189020667 expired; onlineServers=0 2023-07-12 19:12:10,859 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase20.apache.org,43565,1689189019651' ***** 2023-07-12 19:12:10,859 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-12 19:12:10,859 DEBUG [M:0;jenkins-hbase20:43565] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3698cb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-07-12 19:12:10,860 INFO [M:0;jenkins-hbase20:43565] regionserver.HRegionServer(1144): stopping server jenkins-hbase20.apache.org,43565,1689189019651 2023-07-12 19:12:10,860 INFO [M:0;jenkins-hbase20:43565] regionserver.HRegionServer(1170): stopping server jenkins-hbase20.apache.org,43565,1689189019651; all regions closed. 2023-07-12 19:12:10,860 DEBUG [M:0;jenkins-hbase20:43565] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 19:12:10,860 DEBUG [M:0;jenkins-hbase20:43565] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-12 19:12:10,860 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-12 19:12:10,860 DEBUG [M:0;jenkins-hbase20:43565] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-12 19:12:10,860 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1689189021697] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1689189021697,5,FailOnTimeoutGroup] 2023-07-12 19:12:10,860 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1689189021704] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1689189021704,5,FailOnTimeoutGroup] 2023-07-12 19:12:10,860 INFO [M:0;jenkins-hbase20:43565] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-12 19:12:10,860 INFO [M:0;jenkins-hbase20:43565] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-12 19:12:10,860 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-07-12 19:12:10,860 INFO [M:0;jenkins-hbase20:43565] hbase.ChoreService(369): Chore service for: master/jenkins-hbase20:0 had [] on shutdown 2023-07-12 19:12:10,860 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 19:12:10,861 DEBUG [M:0;jenkins-hbase20:43565] master.HMaster(1512): Stopping service threads 2023-07-12 19:12:10,861 INFO [M:0;jenkins-hbase20:43565] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-12 19:12:10,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-12 19:12:10,861 ERROR [M:0;jenkins-hbase20:43565] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-07-12 19:12:10,861 INFO [M:0;jenkins-hbase20:43565] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 19:12:10,861 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-12 19:12:10,862 DEBUG [M:0;jenkins-hbase20:43565] zookeeper.ZKUtil(398): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-07-12 19:12:10,862 WARN [M:0;jenkins-hbase20:43565] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-12 19:12:10,862 INFO [M:0;jenkins-hbase20:43565] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 19:12:10,862 INFO [M:0;jenkins-hbase20:43565] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-12 19:12:10,862 DEBUG [M:0;jenkins-hbase20:43565] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-12 19:12:10,862 INFO [M:0;jenkins-hbase20:43565] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 19:12:10,862 DEBUG [M:0;jenkins-hbase20:43565] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 19:12:10,862 DEBUG [M:0;jenkins-hbase20:43565] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-12 19:12:10,862 DEBUG [M:0;jenkins-hbase20:43565] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 19:12:10,862 INFO [M:0;jenkins-hbase20:43565] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.83 KB heapSize=719.70 KB 2023-07-12 19:12:10,874 INFO [M:0;jenkins-hbase20:43565] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.83 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/7f2c4d422b2549fc92d54f6b8e5b6af9 2023-07-12 19:12:10,877 INFO [M:0;jenkins-hbase20:43565] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7f2c4d422b2549fc92d54f6b8e5b6af9 2023-07-12 19:12:10,877 DEBUG [M:0;jenkins-hbase20:43565] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/7f2c4d422b2549fc92d54f6b8e5b6af9 as hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7f2c4d422b2549fc92d54f6b8e5b6af9 2023-07-12 19:12:10,880 INFO [M:0;jenkins-hbase20:43565] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7f2c4d422b2549fc92d54f6b8e5b6af9 2023-07-12 19:12:10,880 INFO [M:0;jenkins-hbase20:43565] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43855/user/jenkins/test-data/aec5f616-5ef9-1538-b4de-af38af3dae99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7f2c4d422b2549fc92d54f6b8e5b6af9, entries=156, sequenceid=1282, filesize=34.9 K 2023-07-12 19:12:10,881 INFO [M:0;jenkins-hbase20:43565] regionserver.HRegion(2948): Finished flush of dataSize ~608.83 KB/623440, heapSize ~719.68 KB/736952, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 19ms, sequenceid=1282, compaction requested=false 2023-07-12 19:12:10,881 INFO [M:0;jenkins-hbase20:43565] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 19:12:10,881 DEBUG [M:0;jenkins-hbase20:43565] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 19:12:10,884 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 19:12:10,884 INFO [M:0;jenkins-hbase20:43565] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-12 19:12:10,885 INFO [M:0;jenkins-hbase20:43565] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:43565 2023-07-12 19:12:10,886 DEBUG [M:0;jenkins-hbase20:43565] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase20.apache.org,43565,1689189019651 already deleted, retry=false 2023-07-12 19:12:10,958 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 19:12:10,958 INFO [RS:2;jenkins-hbase20:38353] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase20.apache.org,38353,1689189020667; zookeeper connection closed. 2023-07-12 19:12:10,958 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): regionserver:38353-0x100829769890003, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 19:12:10,958 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3abdfcd0] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3abdfcd0 2023-07-12 19:12:10,959 INFO [Listener at localhost.localdomain/40405] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-07-12 19:12:11,058 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 19:12:11,058 INFO [M:0;jenkins-hbase20:43565] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase20.apache.org,43565,1689189019651; zookeeper connection closed. 2023-07-12 19:12:11,059 DEBUG [Listener at localhost.localdomain/40405-EventThread] zookeeper.ZKWatcher(600): master:43565-0x100829769890000, quorum=127.0.0.1:63235, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 19:12:11,063 WARN [Listener at localhost.localdomain/40405] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-12 19:12:11,069 INFO [Listener at localhost.localdomain/40405] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-12 19:12:11,178 WARN [BP-1020377755-148.251.75.209-1689189016963 heartbeating to localhost.localdomain/127.0.0.1:43855] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1020377755-148.251.75.209-1689189016963 (Datanode Uuid 9225ce29-b06f-40b1-aaea-24784a80c6f4) service to localhost.localdomain/127.0.0.1:43855 2023-07-12 19:12:11,181 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518/dfs/data/data5/current/BP-1020377755-148.251.75.209-1689189016963] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 19:12:11,182 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518/dfs/data/data6/current/BP-1020377755-148.251.75.209-1689189016963] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 19:12:11,183 WARN [Listener at localhost.localdomain/40405] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-12 19:12:11,185 INFO [Listener at localhost.localdomain/40405] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-12 19:12:11,208 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 19:12:11,292 WARN [BP-1020377755-148.251.75.209-1689189016963 heartbeating to localhost.localdomain/127.0.0.1:43855] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-12 19:12:11,292 WARN [BP-1020377755-148.251.75.209-1689189016963 heartbeating to localhost.localdomain/127.0.0.1:43855] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1020377755-148.251.75.209-1689189016963 (Datanode Uuid 20bbb2fd-7c0b-4edb-836c-47234fa474c0) service to localhost.localdomain/127.0.0.1:43855 2023-07-12 19:12:11,293 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518/dfs/data/data3/current/BP-1020377755-148.251.75.209-1689189016963] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 19:12:11,293 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518/dfs/data/data4/current/BP-1020377755-148.251.75.209-1689189016963] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 19:12:11,294 WARN [Listener at localhost.localdomain/40405] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-12 19:12:11,297 INFO [Listener at localhost.localdomain/40405] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-12 19:12:11,403 WARN [BP-1020377755-148.251.75.209-1689189016963 heartbeating to localhost.localdomain/127.0.0.1:43855] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-12 19:12:11,403 WARN [BP-1020377755-148.251.75.209-1689189016963 heartbeating to localhost.localdomain/127.0.0.1:43855] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1020377755-148.251.75.209-1689189016963 (Datanode Uuid 1de994d2-d43c-471e-8302-bc00c9b9674d) service to localhost.localdomain/127.0.0.1:43855 2023-07-12 19:12:11,404 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518/dfs/data/data1/current/BP-1020377755-148.251.75.209-1689189016963] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 19:12:11,405 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/af3f9619-5b87-a46b-f30b-f4dd3ee647a3/cluster_708a946f-51ef-94cd-2651-b6a081413518/dfs/data/data2/current/BP-1020377755-148.251.75.209-1689189016963] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 19:12:11,429 INFO [Listener at localhost.localdomain/40405] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-07-12 19:12:11,610 INFO [Listener at localhost.localdomain/40405] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-12 19:12:11,831 INFO [Listener at localhost.localdomain/40405] hbase.HBaseTestingUtility(1293): Minicluster is down